KobitaLM - Bengali Poetry Language Model
KobitaLM (কবিতা LM, meaning "Poetry LM" in Bengali) is a fine-tuned language model specialized in generating Bengali poetry. Built on top of Google's Gemma 2 9B model using LoRA adapters, it captures the distinctive styles of classical and modern Bengali poets.
Model Details
Model Description
KobitaLM is a LoRA adapter fine-tuned on a comprehensive corpus of Bengali poetry from renowned poets including Rabindranath Tagore, Jibanananda Das, Kazi Nazrul Islam, and many others. The model can generate poetry in various styles and follows instructions in Bengali.
- Developed by: Community Project
- Model type: Causal Language Model with LoRA Adapters
- Language: Bengali (বাংলা)
- License: Gemma License
- Base Model: unsloth/gemma-2-9b-bnb-4bit
- Fine-tuning Method: LoRA (Low-Rank Adaptation)
- Primary Use: Bengali poetry generation
Model Sources
- Base Model: Gemma 2 9B
- Training Framework: Unsloth
- Dataset: Bengali Poems Collection (Classical and Modern)
Uses
Direct Use
KobitaLM is designed for:
- Generating Bengali poetry in various classical and modern styles
- Educational purposes to learn about Bengali poetic traditions
- Creative writing assistance for Bengali content creators
- Research in Bengali NLP and computational creativity
Example Usage
1from unsloth import FastLanguageModel
2
3# Load the model
4model, tokenizer = FastLanguageModel.from_pretrained(
5 model_name="your-username/KobitaLM",
6 max_seq_length=2048,
7 dtype=None,
8 load_in_4bit=True,
9)
10
11# Prepare for inference
12FastLanguageModel.for_inference(model)
13
14# Generate poetry
15prompt = "জীবনানন্দ দাশের স্টাইল-এ একটি কবিতা লিখুন।"
16inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
17
18outputs = model.generate(
19 **inputs,
20 max_new_tokens=512,
21 temperature=0.8,
22 top_p=0.9,
23 repetition_penalty=1.2,
24)
25
26poem = tokenizer.decode(outputs[0], skip_special_tokens=True)
27print(poem)
Out-of-Scope Use
This model is NOT suitable for:
- General Bengali text generation outside of poetry
- Translation tasks
- Factual question answering
- Any use requiring factual accuracy or real-time information
Bias, Risks, and Limitations
- The model may reflect biases present in classical Bengali poetry
- Generated content may occasionally contain repetitive patterns
- Performance is optimized for poetry generation, not general text
- Requires Bengali language understanding to evaluate output quality
- May not accurately represent all regional Bengali dialects
Recommendations
- Review generated poetry for appropriateness before publication
- Use as a creative assistant rather than autonomous content generator
- Consider cultural context when using generated poetry
- Respect copyright and attribution norms for the training data poets
Training Details
Training Data
The model was fine-tuned on a curated corpus of Bengali poetry including works from:
- রবীন্দ্রনাথ ঠাকুর (Rabindranath Tagore)
- জীবনানন্দ দাশ (Jibanananda Das)
- কাজী নজরুল ইসলাম (Kazi Nazrul Islam)
- সুকান্ত ভট্টাচার্য (Sukanta Bhattacharya)
- কামিনী রায় (Kamini Roy)
- মাইকেল মধুসূদন দত্ত (Michael Madhusudan Dutt)
- লালন ফকির (Lalon Fakir)
- রামপ্রসাদ সেন (Ramprasad Sen)
- And many other classical and modern Bengali poets
Additionally, Bengali instruction-following data was included for better controllability.
Training Procedure
- Fine-tuning Framework: Unsloth + PEFT
- Training Method: Supervised Fine-Tuning (SFT) with LoRA
- LoRA Rank: 8
- LoRA Alpha: 8
- Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Quantization: 4-bit (bitsandbytes)
- Max Sequence Length: 2048 tokens
- Base Model: unsloth/gemma-2-9b-bnb-4bit
Training Procedure
Preprocessing [optional]
[More Information Needed]
Training Hyperparameters
- Training regime: [More Information Needed]
Speeds, Sizes, Times [optional]
[More Information Needed]
Evaluation
Testing Data, Factors & Metrics
Testing Data
[More Information Needed]
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Model Examination [optional]
[More Information Needed]
Environmental Impact
Carbon emissions can be estimated using the
Machine Learning Impact calculator presented in
Lacoste et al. (2019).
- Hardware Type: [More Information Needed]
- Hours used: [More Information Needed]
- Cloud Provider: [More Information Needed]
- Compute Region: [More Information Needed]
- Carbon Emitted: [More Information Needed]
Technical Specifications [optional]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
[More Information Needed]
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
[More Information Needed]
Framework versions
