The largest open monolingual Assamese corpus for LLM pretraining.
Documents: 2.37M
Characters: 11B
GPT-2 tokens: ~3.7B | Gemma 4 E2B tokens: ~5.8B
Format: Parquet (text column only)
License: ODC-By 1.0
ds = load_dataset("ananddey/asm-corpus", split="train", streaming=True)
for doc in ds:
print(doc["text"])
All documents passed through language… See the full description on the dataset page:
https://huggingface.co/datasets/ananddey/asm-corpus.
