Blog announcement

This repo contains gguf variants of Baguettotron, a 321 million parameters generalist Small Reasoning Model, trained on 200 billions tokens from SYNTH, a fully open generalist dataset.

Despite being trained on consideraly less data, Baguettotron outperforms most SLM of the same size range on non-code industry benchmarks, providing an unprecedented balance between memory, general reasoning, math and retrieval performance.

Please refer to the original model card for more details.

The GGUF conversion was originally done by typeof.

This versions adds a few additional metadata (preferred temperature/settings) to ease the implementation in LM Studio.

Given the tight size of the original model, we recommend to use at least the 8bit version.

We dont provide quants lower than 4bit but they are available in typeof repo.

Easiest installation is through LM Studio, with the default import system.

Otherwise, you can install llama.cpp directly and invoke the llama.cpp server:

or the command line: