You can deploy this model and start earning money today!

yeezer/
gpt2-tamil-merged

Name: gpt2-tamil-merged
Availability: InStock

safetensors

gpt2

velkadamban/Tamil_Wiki_Source

NLPC-UOM/document_alignment_dataset-Sinhala-Tamil-English

openai-community/gpt2-medium

finetune

Views

No views yet

Model Card: GPT-2 Medium Tamil Merged (yeezer/gpt2-tamil-merged)

Model Summary

gpt2-tamil-merged is a 345M-parameter GPT-2 Medium–based model adapted for Tamil and code-mixed Tamil–English text generation. It is produced by merging a standard GPT-2 Medium checkpoint with Tamil-focused training to improve fluency, morphological handling, and contextual coherence in low-resource language settings.

This model is intended as a strong baseline for Tamil text generation and downstream fine-tuning tasks.

Intended Use

Tamil text generation
Code-mixed (Tanglish) generation
Dialogue and story continuation
Research on low-resource language modeling
Fine-tuning for Tamil NLP tasks

Not suitable for high-stakes factual tasks or safety-critical applications.

Model Architecture and Configuration

Base model: GPT-2 Medium Parameters: ~345M Layers: 24 Attention heads: 16 Embedding size: 1024 Context length: 1024 tokens Precision: FP16 Tokenizer: GPT-2 BPE (50,257 tokens)

Configuration snapshot:

json

{
  "model_type": "gpt2",
  "n_layer": 24,
  "n_head": 16,
  "n_embd": 1024,
  "n_positions": 1024,
  "activation_function": "gelu_new",
  "attn_pdrop": 0.1,
  "embd_pdrop": 0.1,
  "resid_pdrop": 0.1,
  "initializer_range": 0.02,
  "layer_norm_epsilon": 1e-05,
  "vocab_size": 50257,
  "bos_token_id": 50256,
  "eos_token_id": 50256,
  "torch_dtype": "float16",
  "use_cache": true,
  "predict_special_tokens": true
}

Training and Merging Details

The model is created using:

GPT-2 Medium as the base architecture
Tamil-adapted GPT-2 derivative checkpoints
A merge procedure to preserve GPT-2 Medium’s general capabilities while improving Tamil fluency

A short stabilization phase was performed after merging to improve consistency and reduce generation drift.

Usage Example

python

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "yeezer/gpt2-tamil-merged"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "தமிழில் ஒரு குறுகிய கதை எழுதுங்கள்:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Strengths

Produces coherent Tamil text
Handles Tamil–English code-mixing effectively
Suitable for creative generation tasks
Easy to fine-tune for specialized use cases

Limitations

May hallucinate or produce inaccurate information
Can generate biased or unsafe content
GPT-2 architecture limits reasoning ability
GPT-2 tokenizer is not optimized for Tamil; subword fragmentation occurs

Safety and Ethical Considerations

The model may output:

Biased or offensive text
Politically sensitive or culturally inappropriate content
Unverified or incorrect information

Filtering is recommended before deployment in user-facing contexts.

License

The model inherits the GPT-2 license from OpenAI and must follow dataset-specific licenses where applicable.

Acknowledgements

Thanks to contributors and open-source Tamil NLP datasets that made this model possible.

Citation

@misc{yeezer2025gpt2tamilmerged,
  title={GPT-2 Medium Tamil Merged Model},
  author={Yeshwanth Kumar},
  year={2025},
  howpublished={GitHub or HuggingFace repository}
}