Code Search-Query Generator (SmolLM2 360M)
This is a
full-parameter supervised fine-tune of
HuggingFaceTB/SmolLM2-360M-Instruct for one narrow,
schema-bound developer-agent subroutine:
Write an executable search regex from a plain-words request.
The model is one cell from the
Parameter Floors for Developer-Agent Subroutines
experiment. Labels are generated by deterministic oracles over real Python
repositories; no teacher model or human judge labels the data.
Intended Use
Use this checkpoint inside the repository's verified subroutine harness, which
renders the task-specific prompt, parses strict JSON, permits one localized
schema-feedback retry, applies deterministic guards, and falls back to rules
where appropriate. This is not a general coding assistant or chat model.
Evaluation
Evaluation uses up to 250 examples from HTTPX and Jinja2, both held out
entirely from training. Decoding is greedy.
| Metric | Result |
|---|
| Success after one schema retry | 62.8% |
| First-pass success | 62.8% |
| First-pass schema validity | 100.0% |
| Base instruct success after retry | 0.0% for the base instruct model |
| Rules-only success | 23.7% |
Experiment verdict for this subroutine: unsolved (best 0.88 @ qwen2.5-0.5b).
Training
- Training examples: 2000
- Epochs: 3.0
- Learning rate: 2e-05
- Effective batch configuration: 32 per device x
1 gradient accumulation
- Maximum sequence length: 2048
- Seed: 0
- Final training loss: 0.409493
- Reproduction hardware: one NVIDIA A100 80GB PCIe
- Source revision:
d0fd7bf
The dataset was generated from pinned Flask, Click, and Rich repositories for
training/validation. HTTPX and Jinja2 were reserved for testing.
Limitations
The checkpoint is specialized to one closed JSON schema and should not be
expected to retain broad instruction-following ability. The experiment mixes
two base-model families across its size sweep. Some subroutines are better
served by deterministic rules; consult the verdict above before deployment.
License
Apache-2.0, following the base model. Experiment code is MIT licensed.
