Model Card for Vian ai Fine-Tuned
Model Details
Model Description
- Developed by: Glavinnguyen
- Model type: Text Generation / Multimodal (Text/Vision/Audio Interface)
- Language(s) (NLP): Vietnamese (Primary), English (Primary)
- License: mit
- Finetuned from model: google/gemma-3-4b-it
Model Sources
- Repository: [More Information Needed]
Uses
Direct Use
This model is specifically fine-tuned on a small-scale dataset to standardize response formatting, style, and structure. It natively retains the advanced deep reasoning mechanisms (<|think|>) and inherent logical problem-solving capabilities of the base Gemma 3 architecture.
Out-of-Scope Use
The model should not be used for tasks requiring extensive broad-domain knowledge expansion outside the scope of the training dataset without human supervision. The fine-tuning process was focused on structuring behavioral output rather than massive knowledge injection.
Training Details
Training Data
The model was trained on a highly curated, high-quality alignment dataset.
Training Procedure
Training was conducted utilizing the QLoRA (Quantized Low-Rank Adaptation) method to minimize hardware resource consumption while aggressively preserving the base model's pre-trained weights.
Training Hyperparameters
The hyperparameters were carefully optimized for an ultra-small dataset to prevent catastrophic overfitting and achieve an ideal convergence point:
- Training regime: QLoRA (FP16/BF16 Mixed Precision)
- Learning Rate: 2e-4 to 1e-5 (Low Learning Rate)
- Per Device Train Batch Size: 1
- Gradient Accumulation Steps: 32 (Global Batch Size = 32)
- Number of Train Epochs: 1 to 2
- Max Length: 256 - 2048 tokens (Allocated to safeguard the generation of
<|think|> tokens)
- Optimizer: AdamW
- LR Scheduler Type: Cosine / Constant
- LoRA Rank (r): 4 or 8
- LoRA Alpha ($\alpha$): 8 or 16
Speeds, Sizes, Times
- Final Training Loss:
0.69 (The sweet spot convergence for low-sample fine-tuning—balancing structural alignment with base intelligence retention).
- VRAM Consumption: ~2.2 GB (When running on a 4-bit Q4_0 execution profile).
Technical Specifications
Model Architecture and Objective
Built upon Google's next-generation Gemma 3 architecture, featuring integrated Quantization Aware Training (QAT) and an intrinsic multi-token prediction (MTP) engine. The model leverages an internal step-by-step reasoning loop before routing structural text outputs via designated generation tags.
Model Card Contact
Framework versions
