This is a English -> Romanian translation model trained by Mihai Popa (with the help of Kimi K2.6 Thinking)! It's based on BART, and it's a lot smaller than other models.

Why?

Because it's for my own future LocalTranslate project! Unlike Google Translate, this uses models (LTR files in the future) that you download and put on your device! And yes, it translates locally!

Notes

Trained in just 22 minutes on Colab T4 GPU. 24M parameters and 92 MB in size!

Model Configurations

Parameter	Value
Tokenizer	BPE
Vocabulary Size	16384 tokens
Batch Size	128 x 1 = 128
Context Window	128 tokens
`max_position_embeddings`	128
`encoder_layers`	6
`decoder_layers`	6
`encoder_attention_heads`	6
`decoder_attention_heads`	6
`encoder_ffn_dim`	768
`decoder_ffn_dim`	768
`d_model`	384
`dropout`	0.1
`attention_dropout`	0.1
`activation_function`	"gelu"
`init_std`	0.02
`scale_embedding`	True
`normalize_before`	True
`add_final_layer_norm`	True
`pad_token_id`	`tokenizer.pad_token_id`
`bos_token_id`	`tokenizer.bos_token_id`
`eos_token_id`	`tokenizer.eos_token_id`
`decoder_start_token_id`	`tokenizer.eos_token_id`
`forced_eos_token_id`	`tokenizer.eos_token_id`
`tie_word_embeddings`	True

Training Configurations

Hyperparameter	Value
`output_dir`	"./localtranslate_en_ro"
`do_train`	True
`do_eval`	True
`eval_strategy`	"steps"
`eval_steps`	500
`learning_rate`	3e-4
`weight_decay`	0.01
`max_steps`	6000
`warmup_steps`	1000
`logging_steps`	50
`save_steps`	500
`save_total_limit`	5000
`fp16`	True
`gradient_checkpointing`	True
`label_smoothing_factor`	0.1
`predict_with_generate`	False
`report_to`	"none"
`dataloader_num_workers`	2
`remove_unused_columns`	False

Limitations

Not Perfect: As with any other model, it's not 100% perfect and can generate incorrect translations!
English-Only: It's for English -> Romanian translation (NOT vice-versa)!

Evaluation Results

Metric	Score (greedy search on dev split)	Score (beam search, 2 beams on dev split)	Score (3 beams on dev split)	Score (greedy on Flores 200)	Score (3-beam on Flores 200)
BLEU	33.74	34.67	34.83	11.31	12.35
chrF++	61.35	62.09	62.29	39.49	40.70

Usage

Code is by Gemini 3 Flash/Kimi K2.6 Thinking (then some little modifications by myself):

python

from transformers import BartForConditionalGeneration, AutoTokenizer
import torch

# 1. Load from your Hugging Face Repo
model_id = "MihaiPopa-1/LocalTranslate_EN_RO"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = BartForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.float32, # Standard for CPU
    device_map="cpu"           # Forces CPU usage
)

# 2. Translate to Romanian
inputs = tokenizer("At your current usage level, this runtime may last up to 1 hour.", return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_length=64,
    num_beams=2,
    early_stopping=True,
    forced_eos_token_id=tokenizer.eos_token_id,
)
translation = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(translation)

Data Used

Dataset	Translation Pairs
Europarl 7	~399k raw -> ~340k cleaned
Tatoeba+	16k
Total	343488
Dev Split	2000