LoRA fine-tune of openbmb/MiniCPM5-1B for Lost Frequency Radio, an interactive radio from parallel universes built for the Hugging Face Build Small Hackathon 2026 (track 🍄 An Adventure in Thousand Token Wood).

Demo: Lost Frequency Radio Space Dataset: ~786 surreal radio transmissions (es / en) with structured tokens, template-generated and hand-curated. Agent trace: the full build trace, scrubbed and shared on the Hub so others can see how it was made.

Task: write short in-character radio scripts (60 to 90 words): 1950s announcers, Jupiter weather reports, impossible commercials, number stations, late-night cross-universe call-in shows.

Two languages on a 1B model: the radio speaks Spanish and English, and the model keeps them apart, no bleeding one into the other. Getting two languages to hold up on a 1-billion-parameter model was the part I most wanted to push, and it worked.

Anti prompt-leak design: the system prompts contain no instruction-shaped rules ("write only the script, 60-90 words..."). The format is learned purely from the completions, so a 1B model has nothing instruction-shaped to "recite" on air.

Structured tokens

The model emits markers that the frontend turns into audiovisual events:

Token	Effect on the radio
`[JINGLE]`	light pulse + arpeggio
`[INTERFERENCIA]`	screen glitch + burst of static
`[CORTE COMERCIAL]`	click + dimming
`[FIN DE TRANSMISION]`	display fade and signal drop

Usage with llama.cpp

python

from llama_cpp import Llama

llm = Llama(model_path="MiniCPM5-1B-lost-frequency-radio-Q4_K_M.gguf", n_ctx=2048)

prompt = (
    "<s><|im_start|>system\nYou are the official voice of the Jupiter Weather "
    "Service, year 2187. You write radio scripts in Spanish.<|im_end|>\n"
    "<|im_start|>user\nWrite tonight's transmission. On-air script only."
    "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
)

tokens = llm.tokenize(prompt.encode(), add_bos=False, special=True)
out = llm.create_completion(prompt=tokens, max_tokens=220, temperature=0.7,
                            stop=["<|im_end|>"])
print(out["choices"][0]["text"])

Note: the <think>\n\n</think>\n\n prefill disables MiniCPM5's reasoning mode (equivalent to enable_thinking=False in the chat template).

Training

786 examples (es / en), LoRA r=16, alpha=32, dropout 0.05, applied to every projection (q/k/v/o/gate/up/down)
3 epochs, lr 1e-4 cosine, bf16, max_length 768
Hardware: a single RTX 4050 laptop (6 GB), the model is tiny by design
Final loss ≈ 0.36-0.42, token accuracy ≈ 0.92

Files

MiniCPM5-1B-lost-frequency-radio-Q4_K_M.gguf: Q4_K_M quantization (~651 MB), the one the Space uses
lora-adapter/: LoRA adapters (to reproduce or continue training)

build-small-hackathon/MiniCPM5-1B-lost-frequency-radio-GGUF

MiniCPM5-1B · Lost Frequency Radio (GGUF Q4_K_M)

Structured tokens

Usage with llama.cpp

Training

Files

build-small-hackathon/
MiniCPM5-1B-lost-frequency-radio-GGUF