from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("asrith05/finetuned_slm_t2_withdomain")
tokenizer = AutoTokenizer.from_pretrained("asrith05/finetuned_slm_t2_withdomain")

# Example: Sports with domain label
prompt = '''Domain: sports
Team1: Lakers, Score1: 108, Team2: Warriors, Score2: 90
Output:'''

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50, temperature=0.8)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
# Expected: "Domain: sports\nTeam1: Lakers, Score1: 108, Team2: Warriors, Score2: 90\nOutput: The Lakers defeated the Warriors with a final score of 108 to 90."

Training Details

Dataset Split: 20k train / 5k validation / 5k test
Epochs: 1 epoch
Learning Rate: 5e-5
Enhanced Vocabulary: 200+ teams, 300+ cities, 400+ names
Format: Domain-aware with structured input-output pairs

Domain-Specific Examples

Sports Domain

Domain: sports
Team1: Mumbai Indians, Score1: 185, Team2: Chennai Super Kings, Score2: 180
Output: Mumbai Indians won against Chennai Super Kings with a score of 185 to 180.

Weather Domain

Domain: weather
City: Hyderabad, Temperature: 32, Condition: sunny, Day: Monday
Output: On Monday, Hyderabad experienced sunny weather with a temperature of 32 degrees.

Travel Domain

Domain: travel
Person: Priya, City: Bangalore, Transport: flight, Duration: 2
Output: Priya traveled to Bangalore by flight, which took 2 hours.

Movies Domain

Domain: movies
Movie: RRR, Genre: Action, Rating: 8.2, Year: 2022
Output: RRR is an Action movie from 2022 with a rating of 8.2.

Products Domain

Domain: products
Product: OnePlus, Brand: OnePlus, Price: 45000, Rating: 4.3
Output: The OnePlus smartphone by OnePlus is priced at ₹45,000 with a rating of 4.3 stars.

Enhanced Vocabulary Features

Global Team Coverage

NBA Teams: Lakers, Warriors, Bulls, Celtics, etc.
IPL Teams: Mumbai Indians, Chennai Super Kings, RCB, etc.
Football Teams: Manchester United, Barcelona, Real Madrid, etc.
Esports Teams: FaZe Clan, Team Liquid, Cloud9, etc.

Worldwide Cities

Indian Cities: Mumbai, Delhi, Bangalore, Hyderabad, Chennai, Pune, Kolkata
Global Cities: New York, London, Tokyo, Paris, Sydney, Dubai, Singapore
Total Coverage: 300+ cities across all continents

Multicultural Names

Indian Names: Priya, Arjun, Ananya, Rohan, Kavya, Aditya
Global Names: John, Emma, Chen, Maria, Ahmed, Sophie
Total Diversity: 400+ names representing multiple cultures

Entertainment Content

Hollywood: Inception, Avengers, The Dark Knight, Titanic
Bollywood: Dangal, Baahubali, 3 Idiots, Zindagi Na Milegi Dobara
Tollywood: RRR, Pushpa, Arjun Reddy, Eega

Key Advantages

Feature	Standard Model	With Domain Labels
Domain Context	Implicit	Explicit Labels
Generation Accuracy	Good	Enhanced
Domain Consistency	Variable	Guaranteed
Training Efficiency	Standard	Domain-Guided
Output Quality	Good	Domain-Optimized

Technical Specifications

Architecture: DeepSeek V3 Compact
Parameters: ~110M trainable parameters
Context Length: 2048 tokens
Training Precision: FP16
Inference Precision: FP32
Optimizer: AdamW with cosine learning rate schedule

Training Process

Data Preparation: 30k diverse examples across 5 domains
Domain Labeling: Explicit domain tags for each example
Vocabulary Enhancement: 5x expanded vocabulary coverage
Balanced Training: Equal representation across all domains
Validation: Continuous validation loss monitoring

Performance Metrics

Training Loss: Converged to 0.27
Validation Loss: Stable at 0.29
Domain Accuracy: >95% correct domain understanding
Fluency Score: High natural language quality
Diversity Index: Enhanced vocabulary utilization

Use Cases

Automated Report Generation: Convert database records to readable reports
Content Creation: Generate diverse content for different domains
Data Storytelling: Transform structured data into narratives
Template-based Generation: Consistent formatting across domains
Multilingual Applications: Support for Indian languages

Model Limitations

Optimized for the specific domain-labeled input format
Best performance on domains similar to training data
Requires well-structured input with clear domain labels
Limited creativity outside of trained domains
May struggle with highly technical or specialized vocabulary

Related Models in the Series

asrith05/finetuned_slm_t2 - No domain labels version
asrith05/slm - Entity extraction specialist
asrith05/deepseek_pretrain_90k - Base pretrained model

Citation

bibtex

@model{finetuned_slm_t2_withdomain,
  title={Fine-tuned SLM T2 With Domain Labels: Multi-Domain Structured Data Generation},
  author={Asrith},
  year={2024},
  url={https://huggingface.co/asrith05/finetuned_slm_t2_withdomain}
}

Acknowledgments

This model builds upon the DeepSeek V3 architecture and was trained on carefully curated diverse datasets. Special thanks to the open-source community for providing the foundational tools and frameworks that made this work possible.

Note: This model is designed specifically for structured data-to-text generation with domain awareness. For best results, always include the domain label in your input format.