Axya-Mini
A fine-tuned GPT-2 adapter model for Dhivehi (Thaana) language question-answering and text generation tasks.
Model Description
Axya-Mini is a lightweight, efficient adapter-based language model specifically designed for the Dhivehi language, the national language of the Maldives. Built on the GPT-2 architecture and optimized using adapter layers, this model excels at question-answering tasks while maintaining compact size and fast inference.
Model Type: Adapter-based Fine-tuned Model
Base Model: GPT-2 (openai-community/gpt2)
Language: Dhivehi (ދިވެހި)
Framework: Quetzal (Revolutionary CPU-optimized training library)
Key Features
- 🌟 Language-Specific: Optimized for Dhivehi (dv) language processing
- ⚡ Lightweight: Efficient adapter architecture for fast inference
- 🎉 Question Answering: Trained on question-answering tasks
- 💾 Safetensors Format: Secure model serialization
- 🤗 Adapter-Based: Uses adapter layers for efficient fine-tuning and storage
Model Details
Intended Use
This model is designed for:
- Question answering in Dhivehi
- Text generation tasks in Dhivehi
- Language understanding for Dhivehi content
- Building Dhivehi NLP applications
Training Data
Dataset: axmeeabdhullo/axya-tech-dv100
A curated Dhivehi dataset containing 100 high-quality technical and educational content samples.
Training Methodology
- Fine-tuning Approach: Adapter-based fine-tuning
- Metrics: Accuracy optimization
- Library: adapter-transformers
- Optimization: Efficient parameter updating through adapter modules
Quetzal Library Optimization
Quetzal is a revolutionary library that powers the efficient training of this model on CPU, making high-quality AI accessible without expensive GPUs.
Library Features:
- 🚀 3x Faster CPU Training: Advanced optimizations for CPU-based training
- 📊 Data Augmentation: Train accurate models with minimal data (5-10x augmentation)
- 💾 Memory Efficient: 4-bit quantization and LoRA for reduced memory usage
- 🎯 High Accuracy: Specialized techniques for low-resource scenarios
- 🌍 Multilingual: Optimized for languages like Dhivehi, but works for any language
- 🔧 Easy to Use: Simple API similar to popular libraries
Installation:
Why Quetzal for Dhivehi?
Quetzal is specifically optimized for low-resource languages like Dhivehi, enabling efficient model training and deployment without requiring expensive GPU infrastructure. This makes it ideal for building NLP models for endangered or underrepresented languages.
Model Performance
- Inference Speed: Fast and efficient due to adapter architecture
- Model Size: Compact compared to full model fine-tuning
- Accuracy: Optimized for Dhivehi language understanding tasks
Limitations
- Trained specifically on Dhivehi language content
- Performance may vary with dialects or regional variations
- Requires GPU/TPU for optimal inference speed
- Limited evaluation on diverse downstream tasks
Recommendations
- Fine-tuning: Can be further fine-tuned on domain-specific Dhivehi data
- Deployment: Use with sufficient computational resources for production
- Evaluation: Test on your specific use case before deployment
- Updates: Check for newer versions of the model for improved performance
Citation
If you use this model, please cite:
1@model{axya_mini,
2 author = {Abdhullo, Axmee},
3 title = {Axya-Mini: Dhivehi Language Question-Answering Model},
4 year = {2025},
5 publisher = {Hugging Face Model Hub},
6 howpublished = {https://huggingface.co/axmeeabdhullo/axya-mini}
7}
License
This model is licensed under the Apache License 2.0. See the LICENSE file for details.
Related Resources
- Dataset: axya-tech-dv100
- Base Model: GPT-2
- Library Documentation: Adapter-Transformers
- Hugging Face Hub: Model Hub
Author
Axmee Abdhullo
AI/ML Developer specializing in Dhivehi NLP
Hugging Face
Contact & Support
For questions, suggestions, or support:
- Open an issue on the model's repository
- Join the Hugging Face community discussions
- Check the model card for updates
Last Updated: December 2024
Status: Active Development
Version: 1.0
