This first release is a research checkpoint trained primarily on aligned English and Iranian Persian Wikipedia articles. It is strongest on Wikipedia-like prose. General-domain translation remains a work in progress; see Evaluation and Limitations.

Usage

Motarjem uses three special control tokens. The prompt format is:

<|end_of_text|><|translate|><|source-language|>SOURCE<|target-language|>

Use <|en|> for English and <|fa|> for Persian. Generation should be greedy for the evaluated behavior.

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Reza2kn/Motarjem-v0.1"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=dtype,
    attn_implementation="sdpa",
).to(device).eval()


def translate(text: str, source_language: str, target_language: str) -> str:
    if (source_language, target_language) not in {("en", "fa"), ("fa", "en")}:
        raise ValueError("Supported directions are en->fa and fa->en")

    prompt_ids = [
        tokenizer.bos_token_id,
        tokenizer.convert_tokens_to_ids("<|translate|>"),
        tokenizer.convert_tokens_to_ids(f"<|{source_language}|>"),
        *tokenizer.encode(text, add_special_tokens=False),
        tokenizer.convert_tokens_to_ids(f"<|{target_language}|>"),
    ]
    input_ids = torch.tensor([prompt_ids], device=device)

    with torch.inference_mode():
        output_ids = model.generate(
            input_ids=input_ids,
            attention_mask=torch.ones_like(input_ids),
            do_sample=False,
            max_new_tokens=256,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.pad_token_id,
            use_cache=True,
        )

    return tokenizer.decode(
        output_ids[0, input_ids.shape[1]:],
        skip_special_tokens=True,
        clean_up_tokenization_spaces=False,
    ).strip()


print(translate("The weather is beautiful today.", "en", "fa"))
print(translate("امروز هوا خیلی خوب است.", "fa", "en"))

Requires a Transformers release with Falcon-H1 support; this checkpoint was trained and evaluated with transformers==4.57.1.

Training

Base model: tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct
Parameters after adding direction tokens: 107,909,824
Dataset: Reza2kn/Wikipedia-FA-EN-DeepSeek-V4-Flash-0731
Pinned dataset revision: 594f3c54d06f2025a04b393cdac3d640e20f432e
Training split: 118,822 aligned article pairs
Frozen holdout: 1,178 article pairs selected deterministically by page ID
Continued pretraining corpus: 114,249,193 tokens
Bidirectional supervised corpus: 194,405,474 tokens across 2,622,774 aligned examples
Maximum training sequence length: 1,024 tokens
Precision: BF16

The stock Falcon tokenizer was retained and extended with <|translate|>, <|en|>, and <|fa|>. A matched custom Persian-tokenizer pilot was more token-efficient but substantially less accurate, so it was not used for this release.

Evaluation

All scores below use greedy decoding, batch size 128, and a maximum of 256 generated tokens.

Suite	Direction	Rows	BLEU	chrF2	TER	Hit token cap
WMT24++	English → Persian	960	3.93	21.87	114.93	13.75%
WMT24++	Persian → English	960	8.03	32.00	103.74	3.75%
Frozen Wikipedia holdout	English → Persian	1,150	41.34	57.16	54.73	2.43%
Frozen Wikipedia holdout	Persian → English	1,150	41.52	59.49	56.21	1.74%

The frozen evaluation uses the 1,150 cleanly materialized rows from the deterministic 1,178-article holdout allocation.

Compared with the untuned official 100M checkpoint on this project's WMT24++ split, Motarjem v0.1 improved chrF2 from 4.13 to 21.87 for English→Persian and from 10.70 to 32.00 for Persian→English. Results from different model families may use different decoding contracts and should not be treated as a strict architecture comparison.

See evaluation/FINAL_EVALUATION_REPORT.md for the full summary and evaluation/failure-analysis.json for categorized failure counts.

Limitations

This is an experimental 108M-parameter checkpoint, not a production translation system.
It is strongly specialized toward Wikipedia-style prose and generalizes poorly to news, social media, chat, short fragments, usernames, and irregular formatting.
On WMT24++, English→Persian produced repetition loops on 13.85% of rows, wrong-language output on 7.08%, and exact source copies on 3.85%. Persian→English was more reliable but still exhibited repetition and number-preservation failures.
Tables, markup, URLs, named entities, and numbers can be brittle.
Long inputs were not validated near the architecture's advertised maximum context. Training used sequences up to 1,024 tokens, and quality degrades on longer held-out inputs.
Machine-generated Wikipedia translations may contain errors or stylistic artifacts inherited from the training data.

Always evaluate on your own domain and retain human review for consequential translations.

Provenance

Final model.safetensors SHA-256: 9ac54ec2c56240ba57ce7e3be0251d1d86acf1ebd125e9381500e30bc1661eb0
The machine-readable training receipt is included as training_receipt.json.
The base model is distributed under the Falcon LLM License; this derivative checkpoint follows that license.

Citation

If you use the Falcon-H1-Tiny family, please cite its creators:

bibtex

@misc{falcon_h1_tiny,
  title  = {Falcon-H1-Tiny: A series of extremely small, yet powerful language models redefining capabilities at small scale},
  author = {Falcon-LLM Team},
  year   = {2026}
}