Views
No views yet
nl2jq-40m: larger, but its
pretrained language priors make it markedly more reliable at copying novel field names
from your JSON — which is what most real CLI use needs.jqgen CLI — the best
latency/accuracy balance for local use (~1–3s per query on laptop CPU).cat data.json | jqgen --backend qwen "total spend per customer, paid orders only"1import re
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4tok = AutoTokenizer.from_pretrained("gauthierpiarrette/nl2jq-qwen3-0.6b")
5model = AutoModelForCausalLM.from_pretrained("gauthierpiarrette/nl2jq-qwen3-0.6b")
6msgs = [{"role": "system", "content": "You translate a natural-language request plus a "
7 "sample of JSON into a single jq program. Output only the jq program, nothing else."},
8 {"role": "user", "content": "Request: highest paid person\nJSON sample: [...]"}]
9enc = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
10ids = enc["input_ids"] # newer transformers return a BatchEncoding here, not a tensor
11out = tok.decode(model.generate(ids, max_new_tokens=128)[0][ids.shape[1]:],
12 skip_special_tokens=True)
13# Qwen3 emits an (empty) reasoning block before the program — strip it:
14print(re.sub(r"(?s)^\s*<think>.*?</think>\s*", "", out).strip())nl2jq-bench v1.0.0
(0% field overlap with training, novel domains, one-shot evaluation). These weights are
the v6 fine-tune — trained on data with per-example-unique field names and coverage of
reduce/foreach/walk/paths/if constructs; the superseded v5 row is kept for the
record:| Model | frozen pass@1 | valid | T1 | T2 | T3 | T4 | T5-gen |
|---|---|---|---|---|---|---|---|
| this model (v6) | 0.40 | 0.73 | 0.65 | 0.52 | 0.36 | 0.28 | 0.20 |
| v5 fine-tune (superseded) | 0.24 | 0.67 | 0.45 | 0.38 | 0.21 | 0.12 | 0.02 |
| Qwen3-0.6B base (zero-shot) | 0.01 | 0.06 | 0.02 | 0.00 | 0.00 | 0.00 | 0.02 |
| nl2jq-qwen3.5-2b (bigger sibling, CLI config) | 0.48 | 0.82 | 0.75 | 0.61 | 0.47 | 0.35 | 0.18 |
| Claude Opus 4.8 (zero-shot, context row) | 0.96 | 0.98 | 1.00 | 0.96 | 0.96 | 0.96 | 0.90 |
reduce/foreach/walk/paths/if-banding constructs. 100% synthetic
and execution-verified — see the
nl2jq dataset. No web text, no
scraped code. Every program was run under jq 1.7.1 and kept only if it produced
non-degenerate output.jqgen CLI prints the program to stderr.
jq runs locally with no network access, but an incorrect program can still mislead.