Masoretic Hebrew to Targumic Aramaic (Onqelos & Jonathan) Translation Model

Model Details

• Model Name: johnlockejrr/marianmt-he2arc-targum
• Base Model: Helsinki-NLP/opus-mt-sem-sem
• Language Pair: Masoretic Hebrew → Targumic Aramaic (he2arc)
• Script: Both languages in Hebrew characters (consonantal)
• Domain: Biblical texts (Torah and Prophets)
• License: MIT

Overview

Key Features

• Base Model: Helsinki-NLP/opus-mt-sem-sem
• Direction: Masoretic Hebrew → Targumic Aramaic (he2arc)
• Script: Both source and target in Hebrew script (consonantal)
• Language Tokens: Uses >>heb<< for Hebrew and >>arc<< for Aramaic
• Optimized: Configured for 12GB GPU with appropriate batch sizes and learning rates

Dataset

• hebrew: Consonantal Hebrew Masoretic Text (Tanakh)
• aramaic: Consonantal Aramaic Targum (Onqelos for Torah, Jonathan for Prophets)
• Additional metadata: book, chapter, verse

Training Configuration

• Batch Size: 8 (optimized for 12GB GPU)
• Learning Rate: 1e-5
• Epochs: 100
• Warmup Steps: 1000
• Gradient Accumulation: 4 (effective batch size: 32)
• FP16: Enabled for faster training
• Language Prefix: Uses >>heb<< and >>arc<< tokens

Performance Metrics

• Final Training Loss: 0.136
• Test Loss: 1.30
• BLEU Score: 42.28
• Character Accuracy: 34.11%
• Vocabulary Size: 33,701
• Model Parameters: 61,917,696

Usage

Inference

1. Single text translation:
   bash

   1python inference.py \ 2 --model_path ./hebrew_aramaic_model_improved \ 3 --text "בראשית ברא אלהים" \ 4 --direction he2arc
2. Using Hugging Face Hub:
   python

   1from transformers import AutoTokenizer, AutoModelForSeq2SeqLM 2 3model_name = "johnlockejrr/marianmt-he2arc-targum" 4tokenizer = AutoTokenizer.from_pretrained(model_name) 5model = AutoModelForSeq2SeqLM.from_pretrained(model_name) 6 7# Translate Hebrew to Aramaic 8text = "בראשית ברא אלהים" 9inputs = tokenizer(f">>heb<< {text}", return_tensors="pt", max_length=512, truncation=True) 10outputs = model.generate(**inputs, max_length=512, num_beams=4) 11translation = tokenizer.decode(outputs[0], skip_special_tokens=True) 12print(f"Translation: {translation}")
3. Batch translation from file:
   bash

   1python inference.py \ 2 --model_path ./hebrew_aramaic_model_improved \ 3 --input_file input_texts.txt \ 4 --output_file translations.txt \ 5 --direction he2arc
4. Interactive mode:
   python inference.py --model_path ./hebrew_aramaic_model_improved

Model Information

• model.safetensors: Model weights
• tokenizer_config.json: Tokenizer configuration
• model_info.json: Training information and metadata
• training_args.bin: Training arguments
• test_results.json: Final evaluation results
• all_results.json: Complete training history

Language Tokens

• >>heb<<: Hebrew (source language)
• >>arc<<: Aramaic (target language)

Requirements

• Python 3.8+
• PyTorch
• Transformers
• Datasets
• Evaluate (for BLEU calculation)
• CUDA-compatible GPU (recommended)

Notes

• The model is specifically optimized for Masoretic Hebrew to Targumic Aramaic translation
• Both source and target are in Hebrew script (consonantal)
• The base model (sem-sem) supports multiple Semitic languages
• Training logs are saved to he2arc_training.log
• The model shows significant improvement with continued training (100 epochs total)
• Best performance achieved with learning rate reduction in continued training phase

Troubleshooting

1. Out of Memory: Reduce batch size or gradient accumulation steps
2. Poor Performance:
   • Check dataset quality
   • Consider continued training with lower learning rate
   • Ensure proper language token usage (>>heb<< and >>arc<<)
3. Language Token Issues: Ensure >>arc<< token is properly added to tokenizer
4. Training Loss Not Decreasing: Try continued training with reduced learning rate (5e-6)
5. Mixed Language Output: This indicates under-training; continue training for more epochs

Citation