NeuroBait is a LoRA fine-tune of unsloth/gemma-3-12b-it for ADHD and
neurodivergent task-initiation conversations.
It is designed for the moment when a person knows what they need to do, but the
first move still feels too heavy. The model aims to respond with warm, short,
agency-preserving language rather than shame, pressure, or a generic productivity
script.
NeuroBait is submitted for the Build Small Hackathon.
Primary track: Backyard AI
Why this track: NeuroBait focuses on a specific, real, everyday problem -
ADHD task initiation - and turns a small model into a practical companion for
that moment.
Bonus quest fit: Well-Tuned, because this repo publishes the fine-tuned
LoRA adapter used by the Space.
Bonus quest fit: Off-Brand, because the app uses custom Gradio UI and
anti-shame product copy rather than a default chatbot shell.
Sponsor fit: Modal-powered, because fine-tuning and generation evaluation
were run on Modal GPU infrastructure.
The project follows the hackathon shape: fine-tune a small-enough open model,
publish the model on Hugging Face, and deploy a working Gradio app as a Hugging
Face Space.
Intended Behavior
NeuroBait should:
respond in short, natural prose,
avoid visible labels such as Micro-action, Hook, or Stakes,
avoid guilt framing and productivity shame,
preserve user agency,
ask one light question when context is too sparse,
offer one tiny concrete action when enough context exists.
It should not act as a medical device, diagnostic tool, therapist, emergency
support system, or replacement for professional care.
Training Data
Run #4 used a bilingual Indonesian/English conversational dataset:
270 train conversations
30 eval conversations
multi-turn messages[] format
official NeuroBait system prompt prepended to each example
The dataset is intentionally not included in this model repo.
Checkpoints: save_strategy="no" to avoid the known Unsloth/TRL checkpoint
pickle issue
Training ran on Modal with an H100 80GB GPU.
Deployment
The deployed Space runs on Hugging Face ZeroGPU.
Runtime path:
Gradio Space
transformers + peft
4-bit bitsandbytes NF4 loading
base model: unsloth/gemma-3-12b-it
LoRA adapter: build-small-hackathon/NeuroBait
Unsloth is used for training, not for Space inference. The dense Gemma 3 12B base
was chosen because it deploys cleanly through the standard
transformers + peft path on ZeroGPU.
Run #4 Results
Training completed 102 steps.
Training summary:
train conversations: 270
eval conversations: 30
train loss: 1.7501
eval loss: 1.8844
The loss signal should be treated as a weak training diagnostic for this project.
NeuroBait is primarily evaluated through generated behavior against the base
model.
Generation eval summary over 8 held-out or novel prompts:
base persona average: 2.25 / 4
fine-tuned persona average: 4.0 / 4
base average words: 80.4
fine-tuned average words: 55.1
base label leaks: 5
fine-tuned label leaks: 0
base action-cue responses: 5
fine-tuned action-cue responses: 4
Qualitatively, the fine-tuned adapter produced shorter, more conversational
responses and did not leak literal structure labels, while the base model leaked
labels in 5 of 8 prompts.
