JameaMT-ar-en-350M is a highly efficient, lightweight decoder-only machine translation model designed to translate text from Arabic to English. Built on top of the revolutionary Liquid Foundation Model (LFM2.5-350M) architecture, this model delivers exceptionally fast inference and maintains a minimal memory footprint, making it ideal for edge devices, CPU environments, and large-scale data processing pipelines.
I fine-tuned this model on 1 million high-quality sentence pairs from the OPUS dataset using an NVIDIA H200 GPU.
Why Liquid Foundation Models (LFM2.5)?
Selected LiquidAI/LFM2.5-350M as my base model due to its state-of-the-art hybrid architecture, which provides several massive advantages over traditional Transformer designs:
Radical Memory Efficiency: By replacing heavy self-attention layers with zero-cache convolution blocks and multiplicative gates, LFM reduces the KV cache footprint by up to 90%.
Scalability & Edge Deployment: The architecture has near-constant inference time and memory complexity. This allows it to achieve massive throughput without requiring massive VRAM, making it exceptionally fast on cloud GPUs and accessible on consumer CPUs or NPUs.
Fewer Embedding Layers: The architecture operates with high parameter efficiency. It natively supports Arabic, allowing it to tokenize and process the script effectively without being weighed down by bloated embedding layers.
Training Loop
Trained the model on an NVIDIA H200 GPU, leveraging Mixed Precision (bfloat16) and Gradient Checkpointing to maximize throughput and memory efficiency.
Here is a breakdown of the evaluation metrics and a qualitative analysis demonstrating the model's strengths based on the provided test samples.
1. Decoding the Evaluation Metrics
SacreBLEU (39.79): BLEU measures the exact word and phrase overlap between the model's output and human references. A score approaching 40 is exceptional for Arabic-to-English translation. It indicates that the model frequently generates high-quality, highly understandable translations that require minimal to no human post-editing.
chrF (60.16): The Character n-gram F-score evaluates translations at the character level rather than just the word level. A score above 60 is highly competitive. It indicates that even when the model chooses a different synonym than the human reference, the root meaning, morphology, and semantic intent are closely aligned.
2. Why the Model is Highly Effective
By analyzing the specific outputs, the power of the model becomes clear. It does not just do word-for-word substitution; it understands context, tone, grammar, and even corrects human reference flaws.
A. Perfect Fluency and Idiomatic Accuracy
The model seamlessly handles everyday dialogue, matching human references perfectly without sounding robotic.
Natural Phrasing: Understands English colloquial structure without awkward literal translations.
B. Mastering Tone and Colloquialisms
A strong translation model must capture the feeling of the text, adapting to informal language, slang, or emotional weight.
Source
Model Prediction
Human Reference
Insight
فى رايى لا يوجد اسوأ من شرطى جبان ملعون.
In my mind, there's nothing worse than a goddamn cowardly cop.
There's nothin' worse in my book than a goddamn yellow cop.
Semantic Depth: The model correctly translates the aggressive tone and meaning. "Cowardly" is the exact meaning of the slang "yellow" used in the reference.
هي متبنّية أسم عصري.
She's adopting a modern name.
She's adopted a fashionable name.
Synonym Flexibility: "Modern" and "fashionable" are practically interchangeable here, showing the model's high chrF score in action.
C. Superiority in Formal and Complex Syntax
Perhaps the most impressive indicator of the model's power is its ability to handle dense, formal sentence structures—sometimes even outperforming the provided human references.
Source
Model Prediction
Human Reference
Insight
وينبغي ألا تقدم التصويبات إلا للنص باللغات الأصلية.
Corrections should be submitted only in the original languages.
Corrections should be submitted to the original languages only.
Grammatical Superiority: The model's placement of "only in the original languages" actually flows better in formal English than the reference.
وكثيراً ما يكون ذلك التعاون في شكل وضع إطار مؤسسي لمتابعة مسائل الاستثمار...
Such cooperation is often in the form of the development of an institutional framework for follow-up on investment issues...
However, some agreements establish only a framework for cooperation between the contracting parties.
High Fidelity: The human reference here heavily summarized the text. The model, however, successfully parsed and translated the entire complex legal/institutional thought flawlessly.
Limitations
The Dialect Deficit (MSA vs. Colloquial Arabic): The most significant constraint stems from the dataset's exclusive reliance on Modern Standard Arabic (MSA / Fusha). Consequently, the model experiences a sharp performance degradation when processing regional dialects (e.g., Egyptian, Levantine, Gulf, Maghrebi) or informal, user-generated content. Because real-world applications—such as social media analysis or casual conversational text—frequently blend MSA with localized vernaculars, the model may struggle to accurately parse non-standard syntactic structures or colloquial slang.
Sensitivity to Source Text Noise and Formatting: Because the model relies on the highly structured nature of MSA, it expects a baseline of grammatical and orthographic correctness. Source text noise—such as missing or ambiguous diacritics (Tashkeel) that alter semantic meaning, spelling errors, or artifacts from upstream OCR extraction pipelines—can disproportionately impact the translation output.
Out-of-Distribution Domain Jargon: The model excels within its primary training distribution but may exhibit lower confidence or rely on literal translations when confronted with highly specialized jargon in untouched domains. Rapidly evolving technical, medical, or niche legal terminology may require targeted fine-tuning or integration with a Retrieval-Augmented Generation (RAG) pipeline to maintain high accuracy.
Document-Level Cohesion in Extended Contexts: The current evaluation highlights exceptional sentence-level and short-paragraph accuracy. However, as is common with highly optimized, parameter-efficient architectures, handling extensive, multi-page documents in a single pass may occasionally lead to losses in pronoun resolution or thematic cohesion, necessitating chunking strategies during inference.
How to Use
To run this model, you will need to ensure trust_remote_code=True is enabled so the custom Liquid architecture is loaded properly.
python
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
34model_id ="ramyibrahim/JameaMT-ar-en-350M"56# 1. Load Tokenizer and Model7tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)8if tokenizer.pad_token isNone:9 tokenizer.pad_token = tokenizer.eos_token
1011model = AutoModelForCausalLM.from_pretrained(12 model_id, trust_remote_code=True, dtype=torch.bfloat16, device_map="auto"13)14model.eval()1516# 2. Format the input using the chat template17sample_text ="تعالى قبل انقضاء الشباب وموت الحب"18messages =[{"role":"user","content": sample_text}]1920inputs = tokenizer.apply_chat_template(21 messages, add_generation_prompt=True, return_tensors="pt", return_dict=True22).to(model.device)2324# 3. Generate the translation25with torch.no_grad():26 output_ids = model.generate(27**inputs,28 max_new_tokens=256,29 do_sample=False,30 num_beams=5,31 early_stopping=True,32 length_penalty=1.0,33 pad_token_id=tokenizer.pad_token_id,34 eos_token_id=tokenizer.eos_token_id,35)3637# 4. Extract and print only the generated response38generated_tokens = output_ids[0][inputs["input_ids"].shape[1]:]39translation = tokenizer.decode(generated_tokens, skip_special_tokens=True)4041print(f"Input: {sample_text}")42print(f"Translation: {translation}")
