This model is a fine-tuned version of Gemma-3-1B-it, developed to enhance Turkish language understanding and knowledge capabilities. The model was evaluated on the standard Turkish MMLU (Massive Multitask Language Understanding) benchmark test and compared against the base model.
📊 MMLU Benchmark Evaluation Results
The performance of the model was evaluated using semantic accuracy and option matching on the Turkish MMLU test set.
🏆 Overall Performance Comparison Table
Model Name
Model Type
Turkish MMLU Score (%)
Performance Change
Gemma-3 1B (Base Model)
Base Model
42.74%
-
aliFurkan123/gemma-3-finetune
Fine-Tuned Model
42.86%
+0.12%
⚙️ Test Methodology & Details
Verification Method: Semantic vector similarity and option verification using paraphrase-multilingual-mpnet-base-v2.
