A fine-tuned version of SmolLM3-3B trained on a distillation dataset generated from a proprietary frontier model, quantized to Q4_K_M GGUF for local inference.

File: SmolLM3-3B-4bit.Q4_K_M.gguf — 1.92 GB

Training Details

Base model: HuggingFaceTB/SmolLM3-3B
Fine-tuning method: Supervised Fine-Tuning (SFT)
Dataset: Private distillation dataset; not publicly available
Hardware: Apple M5 Max (128GB unified memory)
Framework: MLX / Unsloth Studio

Usage

llama.cpp

llama serve -hf Spakie/Smolpus-3B-GGUF-Q4:Q4_K_M

Ollama

ollama run hf.co/Spakie/Smolpus-3B-GGUF-Q4:Q4_K_M

llama-cpp-python

python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="Spakie/Smolpus-3B-GGUF-Q4",
    filename="SmolLM3-3B-4bit.Q4_K_M.gguf",
)
llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain what a transformer is."}]
)

Performance

No evals run yet on this fine-tune. Refer to the base model card for SmolLM3-3B benchmark results (may be innaccurate).

Limitations

May reflect stylistic patterns from the distillation source
Not independently evaluated for safety or bias beyond the base model
Generated content may be factually inaccurate; verify important outputs

License

Apache 2.0 (inherited from base model). Fine-tuning dataset is not released.