Part of the SmolLM-TS series — small language models adapted exclusively to telecommunications standards documents, with zero arXiv or web content in the training corpus.

Looking for the instruction-tuned version? See nareshmodina/SmolLM-TS-360M-it

Model Details


Base model	HuggingFaceTB/SmolLM2-360M
Parameters	360M
Training	Full fine-tuning on TeleSpec-Data
Pretraining data	TeleSpec-Data (1.87B tokens)
Context length	4096 tokens
Hardware	3× NVIDIA L40S (48GB)

Training

Full fine-tuning of all model weights on 457,160 packed 4096-token blocks (1.87B tokens) from 38,302 standards documents — 15,054 3GPP (Rel-8 to Rel-19) and 23,248 ETSI documents spanning 15 working groups (2000–2024). Zero arXiv or web content — 100% standards text.

Epochs: 2
Effective batch size: 128 — LR: 5e-5 (cosine with warmup)
Context length: 4096 tokens

Usage

This is a base model — it continues text rather than following instructions. For instruction following, use SmolLM-TS-360M-it.

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id  = "nareshmodina/SmolLM-TS-360M"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model     = AutoModelForCausalLM.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto"
)

prompt  = "The RRC Connection Establishment procedure in LTE is"
inputs  = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

Base model only — does not follow instructions, use SmolLM-TS-360M-it for Q&A
Standards only — strong 3GPP/ETSI knowledge, limited general telecom knowledge
Not for production — intended for research purposes only

Citation

bibtex

@misc{modina2025smollmts,
  author    = {Naresh Modina},
  title     = {SmolLM-TS: Small Language Models for Telecommunications Standards},
  year      = {2025},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/nareshmodina/SmolLM-TS-360M}
}

nareshmodina/SmolLM-TS-360M

SmolLM-TS-360M

Model Details

Training

Usage

Limitations

Links

Citation

nareshmodina/
SmolLM-TS-360M