dragonfly-135m is a 135M-parameter Llama-based decoder-only language model trained from scratch on 2.7B tokens of synthetic data. Training followed the Chinchilla compute-optimal recipe (20 tokens per parameter), using a single pass over Cosmopedia-v2 with packed sequences and per-document attention masking. It is not a particularly interesting model except for the fact that it exists; it serves as a baseline.

Training sequences are packed using a best-fit algorithm with per-document attention masks to prevent cross-document attention leakage.

This is a raw base model with no instruction tuning or safety alignment. It may produce harmful, inaccurate, or incoherent content. Use with caution.

The model architecture configuration and tokenizer are derived from SmolLM2-135M by HuggingFace, licensed under the Apache 2.0 License. See THIRD_PARTY_LICENSES for details. The model weights are original and released under MIT.