from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the modified model
model = AutoModelForCausalLM.from_pretrained("rnnandi/modified_smollm")
tokenizer = AutoTokenizer.from_pretrained("rnnandi/modified_smollm")

# Test with Bangla text
text = "আমি বাংলায় গান গাই"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

Training

Base Model: HuggingFaceTB/SmolLM2-135M
Tokenizer Enhancement: Merged with TituLM Bangla tokenizer
Original Vocabulary Size: 49,152
Enhanced Vocabulary Size: 180,177
Added Tokens: ~131,025 Bangla-specific tokens

This model was created by:

Merging TituLM Bangla tokenizer with SmolLM2 tokenizer
Resizing model embeddings to accommodate new vocabulary
Preserving original model weights and architecture

Citation

If you use this model, please cite both the original SmolLM2 and TituLM:

bibtex

@misc{smollm2,
  title={SmolLM2: A Family of Small Language Models},
  author={HuggingFace Team},
  year={2024},
  url={https://huggingface.co/HuggingFaceTB/SmolLM2-135M}
}

@misc{titulm,
  title={TituLM: A Bangla Language Model},
  author={Hishab Team},
  year={2024},
  url={https://huggingface.co/hishab/titulm-llama-3.2-1b-v2.0}
}

License

This model is released under the Apache 2.0 License, same as the base SmolLM2 model.