Tini-8B-A1B is a fine-tuned version of the hybrid model architecture LiquidAI/LFM2.5-8B-A1B. This model is optimized for Agentic Reasoning, seamlessly combining deep chain-of-thought (CoT), native system function calling capabilities.
The model was Supervised Fine-Tuned (SFT) on a curated mixture of samples balancing deep reasoning and function-calling actions:
To preserve the model's core capabilities while focusing gradient updates entirely on reasoning tracks, the following configurations were applied:
-
General & Contextual Reasoning (Riddles, Nuances, Analysis):
temperature: 0.6 | top_p: 0.95 | top_k: 50 | repetition_penalty: 1.10
-
Mathematics & Technical Coding Tasks:
temperature: 0.35 | top_p: 0.90 | top_k: 40 | repetition_penalty: 1.08
1import torch
2from unsloth import FastLanguageModel
3from transformers import TextStreamer
4
5MODEL_PATH = "./Tini-8B-A1B"
6
7# 1. Load model with 4-bit quantization for VRAM efficiency
8model, tokenizer = FastLanguageModel.from_pretrained(
9 model_name = MODEL_PATH,
10 max_seq_length = 2048,
11 dtype = torch.bfloat16,
12 load_in_4bit = True,
13 trust_remote_code = True
14)
15FastLanguageModel.for_inference(model)
16
17# 2. Set system prompt forcing Vietnamese internal monologue
18messages = [
19 {
20 "role": "system",
21 "content": "Bạn là một trợ lý AI thông minh. BẮT BUỘC phải thực hiện toàn bộ chuỗi suy luận trong thẻ <think> bằng TIẾNG VIỆT để bảo toàn ngữ cảnh văn hóa và tiết kiệm token."
22 },
23 {
24 "role": "user",
25 "content": "Một bể nước đang cạn hoàn toàn. Nếu mở riêng vòi A đầy sau 4 giờ. Mở riêng vòi B (vòi xả) cạn sau 6 giờ. Hỏi nếu mở cả hai vòi cùng lúc thì sau bao lâu đầy được 75% bể?"
26 }
27]
28
29inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
30text_streamer = TextStreamer(tokenizer, skip_prompt=True)
31
32# 3. Generate with streaming output and a 2048 max token limit
33with torch.no_grad():
34 _ = model.generate(
35 input_ids = inputs,
36 streamer = text_streamer,
37 max_new_tokens = 2048,
38 use_cache = True,
39 temperature = 0.6,
40 top_p = 0.95,
41 top_k = 50,
42 repetition_penalty = 1.10
43 )
