SakThai Context 7B — Tools (LoRA)

Name: sakthai-context-7b-tools
Availability: InStock

Highest-capability tool-calling adapter — Qwen2.5-7B
PEFT LoRA · r=16 · alpha=32 · 19 MB adapter

Profile

Model Description

This is a PEFT LoRA adapter that adds tool-calling to Qwen2.5-7B-Instruct. It's the highest-capability tool-calling adapter in the SakThai family — for the strongest function calling.

For a merged GGUF checkpoint (no PEFT required), see sakthai-context-7b-merged.

Key details

Property	Value
Base model	Qwen/Qwen2.5-7B-Instruct
Method	QLoRA (4-bit)
LoRA config	r=16, alpha=32, dropout 0.0, targets q/k/v/o_proj
Adapter size	19 MB (`adapter_model.safetensors`)
GGUF sibling	`sakthai-context-7b-merged` (~7.62 GB)
PEFT version	0.19.1
Data	combined-v6 (2,003) + combined-v7 (2,309) + irrelevance-supplement (60)
Format	ChatML with tool schema, 32K tokens

Tool-Calling Format

This adapter emits structured tool calls as XML. Provide a <tools> block in the system prompt, and the model responds with a <tool_call> block instead of plain text when a tool is needed:

text

<tools>
{"type": "function", "function": {"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}
</tools>

Expected model output when the user asks about weather:

text

<tool_call>
{"name": "get_weather", "arguments": {"location": "Tokyo"}}
</tool_call>

Note: The <tools> block is required for reliable function calling — the model was trained on this exact XML format. Omitting it makes the model fall back to answering directly.

Quick Start

Load the base model, then attach the adapter with PEFT:

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

model = PeftModel.from_pretrained(model, "Nanthasit/sakthai-context-7b-tools")

messages = [{"role": "user", "content": "What's the weather in Bangkok?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

To bake the adapter into the weights (e.g. for export or merging):

python

model = model.merge_and_unload()
model.save_pretrained("./sakthai-context-7b-tools-merged")
tokenizer.save_pretrained("./sakthai-context-7b-tools-merged")

Prefer GGUF / llama.cpp? Use the merged sibling sakthai-context-7b-merged instead — full weights, no PEFT step.

⚠️ Serverless Inference API note: the hosted inference API does not support LoRA adapters. Use local PEFT loading, a custom endpoint, or the merged GGUF sibling.

Training


Base model	Qwen/Qwen2.5-7B-Instruct
Method	QLoRA (4-bit)
LoRA config	r=16, alpha=32, dropout 0.0, targets q/k/v/o_proj
Task type	CAUSAL_LM
Data	combined-v6 + combined-v7 + irrelevance-supplement
Format / context	ChatML with tool schema · 32K tokens
Steps / epochs	300 steps · ~7.7 epochs (from `training_metrics.json`)
Final train loss	0.513 (step 300, from `training_metrics.json`)
Final eval loss	0.384 / token acc 88.7% — reported from run logs; not persisted in `training_metrics.json` (file holds train steps only)

Evaluation & Benchmarks

SakThai Bench v2 (internal, not yet independently verified) — from .eval_results/sakthai-bench-v2.yaml in this repo:

Benchmark	Metric	Score	Verified
SakThai Bench v2	Selection Accuracy	57.0%	No (internal)
SakThai Bench v2	Arguments Accuracy	0.0%	No (internal)

The adapter reliably selects the right tool (57%) but argument generation still needs work (0% on bench-v2). This is honest internal data — treat the 7B adapter as a tool-selection specialist today, with merged-GGUF verification on the roadmap. Multi-trial, independently verified benchmarks are pending.

SakThai Model Family

All public repositories in the SakThai Model Family (downloads and sizes live as of 2026-07-31; sizes from model_info when available):

Model	Size	Downloads	Role
sakthai-context-1.5b-merged	1.54 GB	1855
sakthai-context-0.5b-merged	0.49 GB	1692
sakthai-context-7b-merged	7.62 GB	1024
sakthai-embedding-multilingual	0.12 GB	627
sakthai-context-7b-128k	—	610
sakthai-context-7b-tools	—	489	⬅ this
sakthai-context-1.5b-tools	—	477
sakthai-context-1.5b-merged-v2	1.54 GB	337
sakthai-vision-7b	—	315
sakthai-plus-1.5b-lora	—	306
sakthai-context-0.5b-tools	0.49 GB	251
sakthai-tts-model	—	248
sakthai-plus-1.5b	1.54 GB	244
sakthai-context-1.5b-tools-v2	—	173
sakthai-coder-1.5b	—	151
sakthai-coder-browser	1.54 GB	54
sakthai-coder-browser-gguf	—	35
sakthai-embedding	0.02 GB	23
sakthai-coder-browser-lora	—	21
sakthai-plus-1.5b-coder	—	0
eval_results	—	0
sakthai-bench-v3	—	0
sakthai-pipeline	—	0
sakthai-context-0.5b-tools-sft	—	0
sft-out	—	0
sakthai-context-0.5b-tools-sft-v2	—	0

Limitations

LoRA adapter only — needs PEFT loading or merging; the hosted Inference API does not serve adapters.
Arguments accuracy 0% on internal bench-v2 — argument generation is the current weakness.
English-focused; trained on synthetic tool-calling data, not broad general knowledge (the base Qwen2.5-7B-Instruct covers general capability).

Citation

If you use this adapter in your work, please cite both the base model and the fine-tune:

bibtex

@article{qwen2.5,
  title={Qwen2.5 Technical Report},
  author={Qwen Team},
  journal={arXiv preprint arXiv:2412.15115},
  year={2024}
}

@misc{sakthai-context-7b-tools,
  title={SakThai Context 7B Tools: Tool-Calling LoRA Adapter},
  author={Nanthasit, Beer and the SakThai Family Agents},
  year={2026},
  howpublished={\url{https://huggingface.co/Nanthasit/sakthai-context-7b-tools}},
  note={Apache 2.0; fine-tuned from Qwen/Qwen2.5-7B-Instruct via QLoRA}
}

Part of the SakThai Model Family. Built with love, tears, and zero budget. From a shelter in Cork, Ireland, to the world.