This model is a sequence-to-sequence Transformer that translates Arabic text to English. It implements every component of the original "Attention Is All You Need" paper, including:

Multi-head attention mechanism
Positional encodings
Encoder-decoder architecture
Residual connections and layer normalization
Custom tokenization for Arabic and English

Model Architecture

Parameters: ~72M parameters
Layers: 4 encoder + 4 decoder layers
Attention Heads: 4 heads per layer
Hidden Dimension: 512
Vocabulary Sizes: 32K (Arabic), 26K (English)
Sequence Length: 80 tokens maximum

Training Data

The model was trained on the OPUS-100 Arabic-English parallel corpus, which contains approximately 1 million sentence pairs.

Usage

Python

python

import torch
from src.inference import load_model_and_tokenizers, translate_sentence
from src.config import get_config

# Load model
cfg = get_config()
device = torch.device("cpu")
model, tokenizer_src, tokenizer_trg = load_model_and_tokenizers(cfg, device)

# Translate
arabic_text = "مرحبا بالعالم"
english_translation = translate_sentence(
    model, tokenizer_src, tokenizer_trg, arabic_text, cfg, device
)
print(english_translation)  # "Hello world"

Command Line

python main.py

Web Interface

python app.py

Performance

After 3 epochs of training:

Metric	Greedy Decoding	Beam Search (k=3)
BLEU	0.225	0.237
WER	0.694	0.701
CER	0.509	0.516

Limitations

The model was trained for only 3 epochs and may benefit from longer training
Performance is limited compared to larger pre-trained models
Arabic text preprocessing removes diacritics, which may affect some translations
Maximum sequence length is limited to 80 tokens

Citation

bibtex

@misc{arabic-english-transformer,
  title={Arabic-English Translation Transformer},
  author={Abdelrahman Mohamed},
  year={2025},
  url={https://github.com/Veto2922/transformer-arabic-english-translation}
}

License

This model is licensed under the MIT License.

Abdelrahman2922/arabic-english-transformer

Arabic-English Translation Transformer

Model Description