Llama 3.1 8B - Indian Income Tax Act 1961 (GGUF)
Fine-tuned Llama 3.1 8B Instruct model specialized in the Indian Income Tax Act 1961. Optimized for tax law queries, compliance questions, and section references.
🎯 Model Details
- Base Model: meta-llama/Meta-Llama-3.1-8B-Instruct
- Fine-tuning Method: LoRA (r=64, alpha=64) with Unsloth
- Training Date: 20251201
- Context Length: 4096 tokens
- Format: GGUF (ready for llama.cpp, Ollama, LM Studio, Jan, etc.)
- Specialization: Indian Income Tax Act 1961
📦 Available Quantizations
| File | Size | Use Case | RAM Required | Quality |
|---|
q4_k_m | ~4.5GB | Recommended - Best balance | 6-8GB | ⭐⭐⭐⭐ |
q5_k_m | ~5.5GB | High quality responses | 8-10GB | ⭐⭐⭐⭐⭐ |
q8_0 | ~8GB | Near-original quality | 10-12GB | ⭐⭐⭐⭐⭐ |
f16 | ~15GB | Maximum quality (if available) | 18-20GB | ⭐⭐⭐⭐⭐ |
Quantization Guide
- q4_k_m: Best for most users - good quality, reasonable size
- q5_k_m: Better quality with slight size increase
- q8_0: Minimal quality loss, larger file
- f16: Full precision, largest file
🚀 Quick Start
Using Ollama
-
-
Create Modelfile
cat > Modelfile << 'EOF'
FROM ./llama-tax-act-q4_k_m.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER top_k 40
PARAMETER repeat_penalty 1.1
SYSTEM """You are an expert on the Indian Income Tax Act 1961.
Provide accurate, detailed information about tax regulations, exemptions,
deductions, and compliance requirements. Always cite relevant sections
when applicable."""
EOF
-
Create the model
ollama create llama-tax-act -f Modelfile
-
Run it
ollama run llama-tax-act
Using llama.cpp
Run inference
./llama-cli -m llama-tax-act-q4_k_m.gguf -p "Explain Section 80C deductions:" -n 512
Run inference
./llama-cli -m llama-tax-act-q4_k_m.gguf -p "Explain Section 80C deductions:" -n 512
from llama_cpp import Llama
Load model
llm = Llama(
model_path="./llama-tax-act-q4_k_m.gguf",
n_ctx=4096,
n_threads=8,
n_gpu_layers=35 # Adjust based on your GPU
)
Generate response
output = llm(
"What are the tax implications under Section 54?",
max_tokens=512,
temperature=0.7,
top_p=0.9,
)
print(output['choices']['text'])
💡 Example Queries
Q: What are the deductions available under Chapter VI-A?
Q: Explain Section 80C and its limit for FY 2023-24
Q: What is the difference between Section 80C and 80D?
Q: How is capital gains tax calculated under Section 112A?
Q: What are the exemptions available under Section 10?
📊 Training Details
- Training Hardware: NVIDIA A100 80GB
- Training Time: ~7 minutes (including quantization)
- Dataset: Custom corpus from Income Tax Act 1961
- Epochs: Optimized for convergence
- Learning Rate: 2e-4 with cosine schedule
- Precision: BF16 training, quantized for deployment
⚙️ Technical Specifications
- Architecture: Llama 3.1 (8B parameters)
- Vocabulary: 128,256 tokens
- Max Context: 4096 tokens
- Attention: Grouped-Query Attention (GQA)
- Activation: SwiGLU
- LoRA Rank: 64 (higher than standard for better quality)
📈 Performance
- Inference Speed (q4_k_m on RTX 3090): ~40-50 tokens/sec
- Inference Speed (q4_k_m on M1 Max): ~25-35 tokens/sec
- Quality: Specialized responses with section references
⚠️ Limitations
- Trained on Income Tax Act 1961 as of training date (20251201)
- May not reflect latest amendments after this date
- Should be used as reference only, not legal advice
- Always verify with official sources
📜 License
🙏 Acknowledgments
- Meta AI for Llama 3.1 base model
- Unsloth AI for efficient fine-tuning framework
- ggerganov for llama.cpp and GGUF format
📧 Contact
For questions or issues, please open an issue on the
repository.
🔄 Updates
2025-12-01: Initial release with q4_k_m, q5_k_m, q8_0 quantizations
Disclaimer: This model is for educational and research purposes. Tax laws are complex and subject to change. Always consult qualified tax professionals for advice.
