Masoretic Hebrew (with nikkud) to Targumic Aramaic (with nikkud and shva na) MarianMT Model

Model Details

• Model Name: hebrew_aramaic_model_improved (local directory)
• Base Model: Helsinki-NLP/opus-mt-sem-sem
• Language Pair: Masoretic Hebrew (with nikkud) → Targumic Aramaic (with nikkud)
• Script: Both languages in Hebrew characters (vocalized, no cantillation)
• Domain: Biblical texts (Tanakh, Torah, Prophets)
• License: MIT

Dataset

• Hebrew Source: Vocalized Masoretic Hebrew Tanakh (with nikkud, no cantillation)
• Aramaic Target: Vocalized Targumic Aramaic (Onqelos for Torah, Jonathan for Prophets)
• Alignment: Verse-aligned, covering the entire Tanakh

Training Configuration

• Base Model: Helsinki-NLP/opus-mt-sem-sem
• Batch Size: 8 (per device, gradient accumulation 4)
• Learning Rate: 1e-5
• Epochs: 100
• FP16: Enabled
• Language Prefix: Uses >>heb<< for Hebrew and >>arc<< for Aramaic
• Tokenizer: MarianMT tokenizer with added special tokens for language direction
• Max Input/Target Length: 512
• Eval Steps: 500
• Save Steps: 500
• Warmup Steps: 1000
• Seed: 42

Performance Metrics

• Final Training Loss: 0.8182
• Test Loss: 0.5223
• BLEU Score: 36.99
• Character Accuracy: 27.05%
• Vocabulary Size: 33,701
• Model Parameters: 61,917,696

Usage

Inference

1. Single text translation:
   bash

   1python inference.py \ 2 --model_path ./hebrew_aramaic_model_improved \ 3 --text "בְּרֵאשִׁית בָּרָא אֱלֹהִים אֵת הַשָּׁמַיִם וְאֵת הָאָרֶץ" \ 4 --direction he2arc
2. Using Hugging Face Transformers (local model):
   python

   1from transformers import AutoTokenizer, AutoModelForSeq2SeqLM 2 3model_path = "./hebrew_aramaic_model_improved" 4tokenizer = AutoTokenizer.from_pretrained(model_path) 5model = AutoModelForSeq2SeqLM.from_pretrained(model_path) 6 7# Translate Hebrew to Aramaic 8text = "בְּרֵאשִׁית בָּרָא אֱלֹהִים אֵת הַשָּׁמַיִם וְאֵת הָאָרֶץ" 9inputs = tokenizer(f">>heb<< {text}", return_tensors="pt", max_length=512, truncation=True) 10outputs = model.generate(**inputs, max_length=512, num_beams=4) 11translation = tokenizer.decode(outputs[0], skip_special_tokens=True) 12print(f"Aramaic: {translation}")
3. Batch translation from file:
   bash

   1python inference.py \ 2 --model_path ./hebrew_aramaic_model_improved \ 3 --input_file input_texts.txt \ 4 --output_file translations.txt \ 5 --direction he2arc
4. Interactive mode:
   python inference.py --model_path ./hebrew_aramaic_model_improved

Model Information

• model.safetensors: Model weights
• tokenizer_config.json: Tokenizer configuration
• model_info.json: Training information and metadata
• training_args.bin: Training arguments
• test_results.json: Final evaluation results
• all_results.json: Complete training history

Language Tokens

• >>heb<<: Hebrew (source language)
• >>arc<<: Aramaic (target language)

Requirements

• Python 3.8+
• PyTorch
• Transformers
• Datasets
• Evaluate (for BLEU calculation)
• CUDA-compatible GPU (recommended)

Notes

• The model is specifically optimized for Masoretic Hebrew (with nikkud, no cantillation) to Targumic Aramaic (with nikkud) translation
• Both source and target are in Hebrew script (vocalized, no cantillation)
• The base model (sem-sem) supports multiple Semitic languages
• Training logs are saved to he2arc_training.log
• The model shows significant improvement with continued training (100 epochs total)
• Best performance achieved with learning rate reduction in continued training phase

Troubleshooting

1. Out of Memory: Reduce batch size or gradient accumulation steps
2. Poor Performance:
   • Check dataset quality
   • Consider continued training with lower learning rate
   • Ensure proper language token usage (>>heb<< and >>arc<<)
3. Language Token Issues: Ensure >>arc<< token is properly added to tokenizer
4. Training Loss Not Decreasing: Try continued training with reduced learning rate (5e-6)
5. Mixed Language Output: This indicates under-training; continue training for more epochs

Citation