A 125M parameter Llama-style language model trained from scratch on ~2.6B tokens of IT support data, then instruction-tuned on ~17K examples including RAFT (Retrieval-Augmented Fine-Tuning) data.
"Not in the context" → "I don't have enough information to answer"
Training Data (17,002 records)
Source
Pairs
Method
Cost
StackExchange (filtered IT)
10,372
Direct extraction
$0
Ubuntu IRC + Gemini
2,821
Teacher distillation
$1.50
Casual interactions
732
Seed + Gemini expansion
$0.04
RAFT dataset
3,077
Gemini 3.5 Flash-Lite
$0.40
Total
17,002
~$1.94
RAFT Pairs Breakdown
Type
Count
Behavior taught
Answerable (grounded)
2,306
Answer strictly from provided context
Unanswerable (refusal)
771
Say "not enough information"
Training Details
Setting
Value
Epochs
3
Learning rate
2e-4 (cosine decay)
Batch size
8
Loss masking
Assistant tokens only
Hardware
1x H100
Best val loss
2.099
Val perplexity
8.83
Total SFT cost
~$2.20
Usage
Plain chat
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23model = AutoModelForCausalLM.from_pretrained("applegrew/support-125M-slm-sft")4tokenizer = AutoTokenizer.from_pretrained("applegrew/support-125M-slm-sft")56chat ="<|bos|><|system|>\nYou are a helpful IT support technician.<|eos|>\n<|user|>\nMy VPN keeps disconnecting every 5 minutes<|eos|>\n<|assistant|>\n"7inputs = tokenizer(chat, return_tensors="pt")8outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.7, do_sample=True)9print(tokenizer.decode(outputs[0], skip_special_tokens=False))
RAG / Grounded (RAFT) style
python
1context ="WiFi drops on Ubuntu 22.04. Run 'iwconfig', check Power Management, disable with 'sudo iwconfig wlan0 power off'."2question ="Why does my wifi keep dropping?"34chat =(5"<|bos|><|system|>\nYou are a helpful IT support technician. Answer using ONLY the "6"provided context. If the answer is not in the context, say you do not have enough "7"information to answer.<|eos|>\n"8f"<|user|>\n\n{context}\n\n\nQuestion: {question}<|eos|>\n<|assistant|>\n"9)
