You can deploy this model and start earning money today!

VenkataRamanaKurumallajaddangi/
Telugu6800

Name: Telugu6800
Availability: InStock

text-generation

pytorch

custom-architecture

VenkataRamanaKurumallajaddangi/Telugu

finetune

apache-2.0

Views

No views yet

🤖 Jaddangi AI Telugu 78M Instruct (V2)

Jaddangi AI Telugu 78M Instruct is a custom-built, 78-million parameter Transformer model designed specifically for the Telugu language. Built entirely from scratch, this model features advanced architectural optimizations like Grouped Query Attention (GQA) and SwiGLU activations, making it highly efficient for local hardware (e.g., i3 processors, 12GB RAM).

🚀 Model Details

Developer: Kurumalla Venkataramana (Jaddangi AI)
Model Name: Jaddangi AI Telugu
Model Size: 78 Million Parameters
Architecture: Custom Decoder-only Transformer
Language: Telugu (తెలుగు)
License: Apache 2.0

🧠 Architecture Highlights

Width: d_model = 576
Depth: 12 Layers
Attention: Grouped Query Attention (8 Q-heads, 2 KV-heads)
Activations: SwiGLU
Normalization: RMSNorm
Positional Embeddings: Rotary Positional Embeddings (RoPE)
Tokenizer: Custom 32k SentencePiece Tokenizer

📚 Training Pipeline

Pre-training (Foundation): Trained on a massive corpus of pure Telugu text, including Wikipedia and the C4 dataset, using memory-optimized mixed-precision (FP16/BF16) and gradient accumulation.
Supervised Fine-Tuning (SFT): Fine-tuned on high-quality, logically structured Telugu instruction pairs (derived from the 10xtechnologieS instruction dataset) to adopt a helpful Chatbot persona (మనుషుడు: -> సహాయకుడు:).

💻 How to Use

Because this model uses a custom PyTorch architecture, you must load your class definitions before loading the weights.

python

import torch
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(repo_id="VenkataRamanaKurumallajaddangi/Telugu", filename="Telugu_Model_Instruct_V2.pt")
tok_path = hf_hub_download(repo_id="VenkataRamanaKurumallajaddangi/Telugu", filename="telugu_spm.model")

# Load Weights
model = TeluguGPT(Config)
model.load_state_dict(torch.load(model_path, map_location='cpu')['model_state_dict'])
model.eval()

# Generate
prompt = "మనుషుడు: ఒక మంచి ఆరోగ్య సూత్రం చెప్పండి.\nసహాయకుడు:"
print(model.generate(prompt))

VenkataRamanaKurumallajaddangi/Telugu6800