You can deploy this model and start earning money today!

Views

No views yet

Sokol — Slovenian Assistant LLM (4B, on-device)

Name: Sokol-4B-SLO
Rating: 2.5 (137 reviews)

Sokol (falcon) is a small 4B Slovenian assistant built for fully-local, on-device use (desktop via GGUF / llama.cpp / LM Studio). It chats in fluent Slovenian, translates EN↔SL, reasons through tasks (<think>), and makes native tool / function calls (Qwen3.6 XML format).

It is a text LLM, not a speech model — pair it with your own ASR + TTS to build a local voice assistant (it routes <think> to reasoning_content, so reasoning is never spoken).

This single repo ships both the full-precision weights (safetensors) and quantized GGUFs (under gguf/), so you can run it with transformers, or fully local with llama.cpp / LM Studio.

What it is

Base: SicariusSicariiStuff/Qwen3.5-4B_Abliterated (dense, qwen3_5 arch, natively supported in transformers ≥ 5.6; multimodal-capable).
Phase 2 — CPT (full-parameter): continued pretraining on ~1.78 B tokens of Slovenian text for language adaptation (full-FT, DeepSpeed ZeRO-2, 1 epoch).
Phase 3 — SFT (LoRA): chat + translation + tool-calling + reasoning + identity (~250 k examples).
Deployment: GGUF (desktop, llama.cpp / LM Studio) and full safetensors (transformers).

Files in this repo

path	format	use
`*.safetensors` (+ `config.json`, tokenizer, `chat_template.jinja`)	full BF16	transformers, further fine-tuning, requantization
`gguf/sokol-4b-slo-Q8_0.gguf`	GGUF Q8_0	highest-quality local (largest)
`gguf/sokol-4b-slo-Q6_K.gguf`	GGUF Q6_K	near-lossless, smaller
`gguf/sokol-4b-slo-Q5_K_M.gguf`	GGUF Q5_K_M	balanced
`gguf/sokol-4b-slo-Q4_K_M.gguf`	GGUF Q4_K_M	smallest, fastest

GGUFs are exported with --no-mtp (the multi-token-prediction head is dropped), which is required for the qwen3_5 GDN-hybrid architecture to load cleanly in current llama.cpp.

Run it (GGUF, llama.cpp)

bash

# thinking + native tool calls; --reasoning-budget routes <think> to reasoning_content
llama-server -m gguf/sokol-4b-slo-Q5_K_M.gguf --jinja --reasoning-budget 8192 \
             -c 8192 --temp 0.4 --top-p 0.9

Or open the .gguf directly in LM Studio. For clean, non-thinking output pass --reasoning-budget 0 (useful when driving TTS in a voice stack, so <think> is never spoken).

Capabilities

Slovenian chat — fluent, formal Slovenian conversation.
Translation — bidirectional EN↔SL.
Reasoning — emits <think> traces (distilled from a large Qwen3.8-Max teacher); pass enable_thinking=False for direct answers.
Tool / function calling — native Qwen3.6 XML, e.g. <tool_call><function=get_weather><parameter=city>Ljubljana</parameter></function></tool_call>.
Identity — presents as Sokol, a local Slovenian assistant.

Training data (provenance)

CPT — Slovenian text (~1.68 M docs / ~1.78 B tokens)

dataset	docs	source	license
`sl_wiki`	160,539	Slovenian Wikipedia	CC BY-SA
`sl_fineweb`	1,523,000	FineWeb2 (sl subset)	ODC-BY

SFT — ~250 k examples

dataset	examples	what	source
`sft_chat_sl`	79,324	Slovenian conversation	`cjvt/GaMS-Nemotron-Chat`
`sft_translate`	120,000	bidirectional EN↔SL translation	OPUS-derived
`sft_toolcall_hermes`	5,086	tool calling	Hermes function-calling
`sft_toolcall_apigen_sokol`	4,829	tool calling (length-filtered)	APIGen-MT
`sft_toolcall_toolace`	9,171	tool calling	ToolACE
`qwen38max_reasoning_sl`	31,255	reasoning `<think>`	distilled from Qwen3.8-Max
`identity_sokol_sl`	56	Sokol identity	hand-written

Nothing is truncated: all SFT examples are ≤ 7942 content tokens (cutoff 8192).

Evaluation

Slovenian-LLM-Eval — acc_norm (`cjvt/slovenian-llm-eval`, 200/task)

Sokol (CPT + SFT) vs the abliterated base, same eval, via flywheel-sl:

task	base Qwen3.5-4B	Sokol	Δ
arc_easy	0.565	0.610	+0.045
arc_challenge	0.425	0.450	+0.025
hellaswag	0.460	0.550	+0.090
piqa	0.605	0.665	+0.060
openbookqa	0.400	0.445	+0.045
winogrande	0.525	0.590	+0.065
boolq	0.825	0.825	+0.000
AVG	0.544	0.591	+0.047

Wins 6/7 tasks, ties 1, no regressions. CPT also halved Slovenian perplexity (6.996 → 3.438 on held-out SL text, −50.9%).

Judge track — chat quality (1–5)

Open-ended answers to 32 general Slovenian questions (8 categories), judged by a stronger 27B Slovenian judge model on correctness / helpfulness / language:

metric	base Qwen3.5-4B	Sokol	Δ
jezik (fluency)	4.88	4.97	+0.09
koristnost (helpfulness)	4.84	4.81	−0.03
pravilnost (correctness)	4.78	4.63	−0.16

Sokol clearly wins language fluency and tool-calling (per-category +0.67) and the objective acc_norm; the small aggregate correctness dip comes almost entirely from two n=4 categories (grammar, instruction-format), i.e. 1–2 questions each — noise at this eval size, not a systematic regression. (v1 is 32 questions; per-category deltas need a larger v2 to be reliable.)

⚠️ Disclaimers & license

General assistant. Outputs may be incorrect or fabricated; not professional advice. Verify anything important.
Mixed data-license provenance (Wikipedia CC BY-SA, FineWeb2 ODC-BY, GaMS-Nemotron-Chat, OPUS, Hermes/APIGen/ToolACE, Qwen3.8-Max distillation). Marked license: other; resolve data terms before public/commercial use. Derivative of Qwen3.5-4B — the upstream Qwen license also applies.
Provided AS IS, no warranty. See LICENSES.md.

Citation

@misc{sokol_4b_slovenian_assistant,
  title  = {Sokol: a small on-device Slovenian assistant LLM (4B)},
  author = {Tadej Fius},
  year   = {2026},
  note   = {Derivative of Qwen3.5-4B. Slovenian CPT+SFT. General Slovenian assistant LLM.}
}

texdata/Sokol-4B-SLO