This repository contains GGUF format model files for SARC-Taigi-LLM-12b. This model is a specialized version of google/gemma-3-12b-it, fine-tuned by the Speech AI Research Center (SARC) using IMA's 'Taiwan Tongues' Taigi Datasets and QLoRA.
The GGUF versions are optimized for inference on consumer-grade hardware (CPUs or GPUs with limited VRAM) via llama.cpp, Ollama, or other compatible backends.
1. Main Capabilities
Taigi Dialogue and Consultation: Understands and responds to inquiries in Taigi using both Taiwanese Chinese characters (Hàn-jī) and Romanization (Tâi-lô).
Linguistic Knowledge Retrieval: Supports queries regarding the meaning, usage, and cultural context of Taigi vocabulary.
Logical Reasoning: Capable of complex reasoning and problem-solving within a Taiwanese linguistic and cultural framework.
2. Quantization Versions (GGUF)
For a 12b model, we recommend Q4_K_M for the best balance between speed and linguistic precision.
File Name
Method
RAM/VRAM Required
Description
SARC-Taigi-LLM-12b-Q8_0.gguf
Q8_0
~14-16 GB
Near-lossless quality. Recommended for academic research and high-precision tasks.
SARC-Taigi-LLM-12b-Q6_K.gguf
Q6_K
~12-14 GB
High precision for specialized linguistic research.
SARC-Taigi-LLM-12b-Q4_K_M.gguf
Q4_K_M
~9-11 GB
Highly Recommended. Optimal quality/performance ratio for 12GB GPUs.
SARC-Taigi-LLM-12b-Q3_K_L.gguf
Q3_K_L
~7-9 GB
Lightweight version for memory-constrained or 8GB VRAM devices.
3. Main Capabilities
Taigi Dialogue and Consultation: Capable of understanding and responding to daily and professional inquiries in Taigi (using Taiwanese Chinese characters (Tâi-bûn Hàn-jī) or Romanization (Tâi-lô)).
Linguistic Knowledge Retrieval: Supports queries regarding the meaning, usage, and cultural background of Taigi vocabulary.
Logical Reasoning: Performs logical judgment and problem-solving specifically within a Taigi linguistic context.
The model underwent a two-stage training process designed to build a robust linguistic foundation, followed by instruction alignment:
Phase 1: Continual Pre-Training (CPT)
Ministry of Education Dictionary of Frequently-Used Taiwanese Taigi.
Taigi Literature Collection (taigi-literature): A diverse corpus of classical and modern Taigi literary works.
Phase 2: Supervised Fine-Tuning (SFT)
Taigi-version Alpaca Dataset: Instruction-following data optimized for Taigi dialogue.
Grand Challenge Training Set: Multiple-choice questions from the training text of the 1st "Grand Challenge" (科技大擂台) competition.
6. Evaluation on << 2020 Grand Challenge, Talk to AI >> Final-Test Dataset
We evaluated the models using the << 2020 Grand Challenge, Talk to AI (科技大擂台，與AI對話)>> Final-Test Dataset, which consists of 1,000 multiple-choice reading comprehension questions. This serves as a benchmark for Taigi language understanding:
Question Example
Experimental Results
Model
Accuracy
Note
Stage
Gemma-3-12b-it
Gemma-3-27b-it
Original
0.80320
0.86214
Baseline performance
After CPT
0.88312
0.92296
Knowledge internalization
After SFT
0.89610
0.92582
Instruction alignment
7. Model Usage
via Ollama
bash
1# Ensure Ollama is installed and your Ollama SSH public key (typically found at ~/.ollama/id_ed25519.pub) is registered in your Hugging Face account.23ollama pull huggingface.co/Speech-AI-Research-Center/SARC-Taigi-LLM-12b-GGUF:Q4_K_M
ollama run huggingface.co/Speech-AI-Research-Center/SARC-Taigi-LLM-12b-GGUF:Q4_K_M
ollama ps
ollama run huggingface.co/Speech-AI-Research-Center/SARC-Taigi-LLM-12b-GGUF:Q4_K_M
via llama.cpp
bash
1# Ensure you are in the llama.cpp directory2cd /path/to/llama.cpp
34# Run the Taigi model with optional optimization flags5./build/bin/llama-cli \6 -hf Speech-AI-Research-Center/SARC-Taigi-LLM-12b-GGUF:Q4_K_M \7 -p "<start_of_turn>user\n用台語共我紹介一下你自己。<end_of_turn>\n<start_of_turn>model\n"\8 -n 512\9 -ngl 99\10 --temp 0.7
Optional Arguments:
-n, --n-predict (Default: 128)
Specifies the maximum number of tokens to generate. For a 12b model, we recommend 512 or higher to ensure the Taigi responses are not cut off mid-sentence.
-ngl, --n-gpu-layers (Default: 0)
Crucial for Performance: Determines how many model layers are offloaded to the GPU.
Setting it to 99 (or any number higher than the actual layers) forces the entire model into VRAM for maximum speed.
Note: If your VRAM is insufficient (e.g., less than 20GB for Q4_K_M), decrease this number to perform "partial offloading" to the CPU. If omitted, the model runs entirely on the CPU, which will be significantly slower.
--temp (Default: 0.8)
Adjusts the randomness of the output.
0.7 provides a good balance between creativity and coherence for Taigi dialogue. Use a lower value (e.g., 0.2) for factual tasks.
8. Roadmap: Beyond SFT
While the current release is the result of CPT and SFT, this is only the beginning. Our multi-stage alignment strategy includes:
Phase I (CPT): Building linguistic foundation (Completed).
Phase II (SFT): Instruction and dialogue alignment (Current Release).
Phase III (GRPO): Future reinforcement learning using Group Relative Policy Optimization (GRPO) to further enhance self-correction and complex reasoning chains.
9. Training Resources
Learn how to perform this multi-stage fine-tuning (CPT + SFT) with our custom callbacks for Loss minimization and Gap stability on GitHub:
If you find this project useful, please cite the IMA's Taiwan Tongues resource page and the Speech AI Research Center organization pages on Hugging Face and GitHub.
bibtex
1@misc{ima_taiwan_2026,
2 title = {IMA-Taiwan},
3 author = {Information Management Association of R.O.C. (IMA)},
4 year = {2026},
5 howpublished = {https://huggingface.co/IMA-Taiwan},
6 note = {Hugging Face organization page for Taiwan Tongues resources}
7}
bibtex
1@misc{sarc_hf_2026,
2 title = {Speech-AI-Research-Center},
3 author = {Speech AI Research Center (SARC)},
4 year = {2026},
5 howpublished = {https://huggingface.co/Speech-AI-Research-Center},
6 note = {Hugging Face organization page for released Taigi model adapters}
7}
bibtex
1@misc{sarctaigillm_repo_2026,
2 title = {Speech-AI-Research-Center},
3 author = {Speech AI Research Center (SARC)},
4 year = {2026},
5 howpublished = {https://github.com/Speech-AI-Research-Center},
6 note = {GitHub organization page for released Taigi-LLM training project}
7}
License
This model is subject to the Gemma Terms of Use. By using this model, you agree to comply with Google’s licensing requirements.
