A lightweight, 76-million parameter machine translation model fine-tuned to translate accounting concepts and ERP system manuals from Arabic to English.
Model Details
Model Description
This model is a fine-tuned version of Helsinki-NLP/opus-mt-ar-en, specifically optimized for the financial and enterprise resource planning (ERP) domain. With only 76 million parameters, the model is highly efficient and easily fits on a standard CPU for fast, local inference without requiring specialized GPU hardware.
Developed by: Ramy Ibrahim Jamea
Model type: Transformer-based Machine Translation
Language(s) (NLP): Arabic (ar) to English (en)
License: Apache 2.0 (Inherited from MarianMT/Opus-MT, verify based on your specific datasets)
Finetuned from model:Helsinki-NLP/opus-mt-ar-en
Uses
Direct Use
The model is intended for direct integration into ERP systems, accounting software, or document processing pipelines where Arabic financial documents, invoices, or accounting manuals need to be translated into English accurately.
Out-of-Scope Use
The model is highly specialized in accounting terminology. It may underperform or produce unnatural translations if used for general-purpose conversational translation, literary texts, or highly informal dialects (e.g., social media text).
Bias, Risks, and Limitations
Due to its specialized training data, the model might rigidly force general Arabic words into accounting-specific English contexts. Users should be aware that synthetic data was used in the training pipeline, which occasionally introduces hallucinated terminologies if the base LLM generator drifted from standard definitions.
The model was fine-tuned on a combined dataset consisting of 18,000 parallel Arabic-English sentences specialized in accounting and ERP domains.
Original Data: 14,000 curated samples.
Synthetic Data: 4,000 samples generated via an LLM pipeline. This pipeline utilized 10 random samples from the original dataset as few-shot prompts to generate new pairs that strictly adhered to the target terminology and translation tone.
Training Procedure
Training Hyperparameters
The model was trained using the following configuration:
Optimizer: Fused AdamW (adamw_torch_fused)
Learning rate: 5e-5
Learning rate scheduler: Cosine with a 0.10 warmup ratio
Epochs: 10 (with Early Stopping patience of 3)
Batch size: 16 (Train and Eval)
Max Gradient Norm: 1.0
Gradient Checkpointing: Enabled
Mixed Precision:bf16
Evaluation
Testing Data, Factors & Metrics
Metrics
The primary evaluation metric utilized for determining the best model checkpoint was chrF (Character n-gram F-score), which is well-suited for translation tasks involving morphologically rich languages like Arabic.
Results
Best Validation chrF: 71.19684
Final Training Loss: 0.29289
Final Validation Loss: 0.57393
Training & Evaluation Plots
Loss Curve
Evaluation Curve
chrF Score
Technical Specifications
Model Architecture and Objective
The architecture follows the standard MarianMT (Bart-like encoder-decoder) setup with 76 million parameters, optimized for sequence-to-sequence conditional generation.
