Metric	Value
Valid CadQuery generation	98% (100-prompt benchmark)
Base model	Qwen/Qwen3.5-2B
Method	LoRA rank 32
Adapter size	129 MB
Training data	1000 reasoning-augmented samples

Quick Start

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.5-2B",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="cuda",
)
model = PeftModel.from_pretrained(base, "raspbfox/cad-qw35-2b-0.1")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-2B", trust_remote_code=True)
tok.pad_token = tok.eos_token

prompt = "Create a 60x40x5mm enclosure base with snap-fit posts"
messages = [{"role": "user", "content": prompt}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt", max_length=2048).to("cuda")
out = model.generate(**inputs, max_new_tokens=600, temperature=0.3, top_p=0.9)
print(tok.decode(out[0], skip_special_tokens=True))

Training Details

Base model: Qwen/Qwen3.5-2B (2B parameter vision-language model)
Fine-tuning method: LoRA (rank 32, alpha 64, no bias)
Hardware: Single RTX 3090 24GB
Hyperparameters: lr=1.5e-4, per_device_batch=2, grad_accum=4, bf16, 4 epochs
Loss: Next-token prediction on assistant responses only

Dataset

The training data consists of triples: {prompt, reasoning, code} where:

prompt is a natural language CAD request (e.g. "70x40x6mm enclosure lid")
reasoning is a step-by-step reasoning trace generated by Qwen3.5-9B
code is the corresponding CadQuery Python code from Zero-To-CAD-100k