Views
No views yet
jqgen --backend qwen-2b
option. The LoRA adapter is merged; it loads as an ordinary model.nl2jq-bench v1.0.0
(0% field overlap with training, novel domains, one-shot evaluation — see the bench card):| system | pass@1 | valid | T1 | T2 | T3 | T4 | T5-gen |
|---|---|---|---|---|---|---|---|
| this model, greedy | 0.46 | 0.75 | 0.67 | 0.61 | 0.47 | 0.33 | 0.13 |
| this model + exec-rerank k=4 (the CLI config) | 0.48 | 0.82 | 0.75 | 0.61 | 0.47 | 0.35 | 0.18 |
| nl2jq-qwen3-0.6b-v6 (smaller sibling) | 0.40 | 0.73 | 0.65 | 0.52 | 0.36 | 0.28 | 0.20 |
| Claude Opus 4.8 (zero-shot, context row) | 0.96 | 0.98 | 1.00 | 0.96 | 0.96 | 0.96 | 0.90 |
jqgen runs by default.cat data.json | jqgen --backend qwen-2b "total ridership per garage, highest first"1from transformers import AutoModelForCausalLM, AutoTokenizer
2tok = AutoTokenizer.from_pretrained("gauthierpiarrette/nl2jq-qwen3.5-2b")
3model = AutoModelForCausalLM.from_pretrained("gauthierpiarrette/nl2jq-qwen3.5-2b")
4msgs = [{"role": "system", "content": "You translate a natural-language request plus a "
5 "sample of JSON into a single jq program. Output only the jq program, nothing else."},
6 {"role": "user", "content": 'Request: highest fare\nJSON sample: [{"rider":"a","fare":12}]'}]
7enc = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt",
8 enable_thinking=False)
9ids = enc["input_ids"]
10out = model.generate(ids, max_new_tokens=128,
11 eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"))
12print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True).strip())reduce/foreach/walk/paths/if and friends). All data is synthetic and
execution-verified under jq 1.7.1; no scraped content; no frontier-model outputs.
Development selection used a separate dev split — the frozen benchmark was evaluated
exactly once.--no-run).convert_hf_to_gguf.py produces a loadable file), and the result
loads without errors — but in our testing (llama.cpp b9957/b9964, CPU inference on macOS
and Linux) generation for this hybrid GDN+MoE architecture never completed in reasonable
time. We don't publish artifacts whose output we could not verify. Use the transformers
path above; a GGUF will follow when llama.cpp's kernels for this architecture mature.