Benchmark	Qwen3-4B-Instruct-2507	kosa-4B-it-v1
GSM8K (strict)	73.24%	84.23%
GSM8K (flexible)	79.15%	85.60%
IFEval (prompt strict)	83.36%	85.77%
IFEval (instruction strict)	88.61%	90.29%
ARC-Challenge (acc_norm)	43.09%	52.13%
MMLU	61.89%	65.76%
Average	71.56%	77.30%

Both models evaluated in the same session with identical settings: lm-evaluation-harness 0.4.12, vLLM, bfloat16, temperature 0, chat template applied. Raw result files in /benchmarks. Training data verified clean against all four benchmark test sets (13-gram and 8-gram overlap checks with positive-control validation).

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "kosa-labs/kosa-4B-it-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "Write a concise project update."}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

GGUF quantizations are available in /gguf:

gguf/kosa-4B-it-v1-Q4_K_M.gguf
gguf/kosa-4B-it-v1-Q5_K_M.gguf
gguf/kosa-4B-it-v1-Q8_0.gguf