You can deploy this model and start earning money today!

AraFusion/
arafusion-mdlm-641m-charlevel-scratch-v2

Name: arafusion-mdlm-641m-charlevel-scratch-v2
Availability: InStock

Views

No views yet

AraFusion — Arabic Masked Diffusion Language Model

Trained with MDLM on FineWeb-2 Arabic subsets. Supports dialect-conditioned generation via Classifier-Free Guidance (CFG).

Model details


Architecture	DiT (BERT-base scale: 12L / 12H / 768d)
Vocab	96 000 (MorphBPE, see `AraFusion/arafusion-morphBPE`)
Sequence length	512 tokens
Diffusion	Linear noise schedule, 1 000 sampling steps
Dialects	MSA (`[MSA]`), Najdi (`[NAJDI]`), Egyptian (`[EGYPT]`)
CFG p_uncond	0.10
Training steps	200 000 (pre-train)
Precision	BF16

Data

Trained on three FineWeb-2 Arabic subsets:

Subset	Dialect	Split
`arb_Arab`	Modern Standard Arabic	train 10 %
`ars_Arab`	Najdi / Saudi Arabic	train (full)
`arz_Arab`	Egyptian Arabic	train (full)

See AraFusion/arafusion-arabic-raw for the raw text and AraFusion/arafusion-arabic-packed for the packed training sequences.

Checkpoints

Training job: charlevel-641m-51502

Path in repo
`checkpoints/0-1000-v1.ckpt`
`checkpoints/0-1000-v2.ckpt`
`checkpoints/0-1000.ckpt`
`checkpoints/0-100000.ckpt`
`checkpoints/0-102000.ckpt`
`checkpoints/0-104000.ckpt`
`checkpoints/0-106000.ckpt`
`checkpoints/0-108000.ckpt`
`checkpoints/0-110000.ckpt`
`checkpoints/0-112000.ckpt`
`checkpoints/0-114000.ckpt`
`checkpoints/0-116000.ckpt`
`checkpoints/0-118000.ckpt`
`checkpoints/0-120000.ckpt`
`checkpoints/0-122000.ckpt`
`checkpoints/0-124000.ckpt`
`checkpoints/0-126000.ckpt`
`checkpoints/0-128000.ckpt`
`checkpoints/0-130000.ckpt`
`checkpoints/0-132000.ckpt`
`checkpoints/0-134000.ckpt`
`checkpoints/0-136000.ckpt`
`checkpoints/0-138000.ckpt`
`checkpoints/0-140000.ckpt`
`checkpoints/0-142000.ckpt`
`checkpoints/0-144000.ckpt`
`checkpoints/0-146000.ckpt`
`checkpoints/0-148000.ckpt`
`checkpoints/0-150000.ckpt`
`checkpoints/0-152000.ckpt`
`checkpoints/0-154000.ckpt`
`checkpoints/0-156000.ckpt`
`checkpoints/0-158000.ckpt`
`checkpoints/0-160000.ckpt`
`checkpoints/0-162000.ckpt`
`checkpoints/0-164000.ckpt`
`checkpoints/0-166000.ckpt`
`checkpoints/0-168000.ckpt`
`checkpoints/0-170000.ckpt`
`checkpoints/0-172000.ckpt`
`checkpoints/0-174000.ckpt`
`checkpoints/0-176000.ckpt`
`checkpoints/0-178000.ckpt`
`checkpoints/0-180000.ckpt`
`checkpoints/0-182000.ckpt`
`checkpoints/0-184000.ckpt`
`checkpoints/0-186000.ckpt`
`checkpoints/0-188000.ckpt`
`checkpoints/0-190000.ckpt`
`checkpoints/0-192000.ckpt`
`checkpoints/0-194000.ckpt`
`checkpoints/0-196000.ckpt`
`checkpoints/0-198000.ckpt`
`checkpoints/0-200000.ckpt`
`checkpoints/0-41500-v1.ckpt`
`checkpoints/0-42000.ckpt`
`checkpoints/0-42500.ckpt`
`checkpoints/0-43000.ckpt`
`checkpoints/0-43500.ckpt`
`checkpoints/0-44000.ckpt`
`checkpoints/0-44500.ckpt`
`checkpoints/0-45000.ckpt`
`checkpoints/0-45500.ckpt`
`checkpoints/0-46000.ckpt`
`checkpoints/0-46500.ckpt`
`checkpoints/0-47000.ckpt`
`checkpoints/0-47500.ckpt`
`checkpoints/0-48000.ckpt`
`checkpoints/0-48500.ckpt`
`checkpoints/0-49000.ckpt`
`checkpoints/0-49500.ckpt`
`checkpoints/0-50000-v1.ckpt`
`checkpoints/0-50000.ckpt`
`checkpoints/0-50500.ckpt`
`checkpoints/0-51000.ckpt`
`checkpoints/0-51500.ckpt`
`checkpoints/0-52000.ckpt`
`checkpoints/0-52500.ckpt`
`checkpoints/0-53000.ckpt`
`checkpoints/0-53500.ckpt`
`checkpoints/0-54000.ckpt`
`checkpoints/0-54500.ckpt`
`checkpoints/0-55000.ckpt`
`checkpoints/0-55500.ckpt`
`checkpoints/0-56000.ckpt`
`checkpoints/0-56500.ckpt`
`checkpoints/0-57000.ckpt`
`checkpoints/0-57500.ckpt`
`checkpoints/0-58000.ckpt`
`checkpoints/0-58500.ckpt`
`checkpoints/0-59000.ckpt`
`checkpoints/0-59500.ckpt`
`checkpoints/0-60000.ckpt`
`checkpoints/0-60500.ckpt`
`checkpoints/0-61000.ckpt`
`checkpoints/0-61500.ckpt`
`checkpoints/0-62000.ckpt`
`checkpoints/0-62500.ckpt`
`checkpoints/0-63000.ckpt`
`checkpoints/0-63500.ckpt`
`checkpoints/0-64000.ckpt`
`checkpoints/0-64500.ckpt`
`checkpoints/0-65000.ckpt`
`checkpoints/0-65500.ckpt`
`checkpoints/0-66000.ckpt`
`checkpoints/0-66500.ckpt`
`checkpoints/0-67000.ckpt`
`checkpoints/0-67500.ckpt`
`checkpoints/0-68000.ckpt`
`checkpoints/0-68500.ckpt`
`checkpoints/0-69000.ckpt`
`checkpoints/0-69500.ckpt`
`checkpoints/0-70000.ckpt`
`checkpoints/0-70500.ckpt`
`checkpoints/0-71000.ckpt`
`checkpoints/0-71500.ckpt`
`checkpoints/0-72000.ckpt`
`checkpoints/0-72500.ckpt`
`checkpoints/0-73000.ckpt`
`checkpoints/0-73500.ckpt`
`checkpoints/0-74000.ckpt`
`checkpoints/0-74500.ckpt`
`checkpoints/0-75000.ckpt`
`checkpoints/0-76000.ckpt`
`checkpoints/0-78000.ckpt`
`checkpoints/0-80000.ckpt`
`checkpoints/0-82000.ckpt`
`checkpoints/0-84000.ckpt`
`checkpoints/0-86000.ckpt`
`checkpoints/0-88000.ckpt`
`checkpoints/0-90000.ckpt`
`checkpoints/0-92000.ckpt`
`checkpoints/0-94000.ckpt`
`checkpoints/0-96000.ckpt`
`checkpoints/0-98000.ckpt`
`checkpoints/best.ckpt`
`checkpoints/last-v1.ckpt`
`checkpoints/last.ckpt`

Usage

python

# Requires the MDLM library from https://github.com/kuleshov-group/mdlm
from transformers import PreTrainedTokenizerFast
tok = PreTrainedTokenizerFast.from_pretrained("AraFusion/arafusion-morphBPE")

# Load checkpoint and run a sampling step — see MDLM docs for full API.

Citation

bibtex

@misc{arafusion2026,
  title   = {AraFusion: Dialect-Conditioned Arabic Masked Diffusion Language Model},
  year    = {2026},
  url     = {https://huggingface.co/AraFusion}
}

AraFusion/arafusion-mdlm-641m-charlevel-scratch-v2