netcfg-8b-lora
A LoRA adapter for Qwen3-8B that converts raw network device configuration
text into a normalized JSON object, across 17 vendor platforms.
Trained entirely on a single laptop (AMD Ryzen AI Max+ 395 / Strix Halo) with no
cloud compute and no proprietary data.
Turkce ozet: Ag cihazi konfigurasyonlarini normalize JSON'a ceviren, Qwen3-8B
uzerine egitilmis bir LoRA adaptoru. 17 farkli uretici platformu kapsiyor. Tamamen
tek bir dizustu bilgisayarda, bulut kullanmadan egitildi.
What it does
Input: a device configuration (Cisco IOS/IOS-XR/NX-OS, Arista EOS, Nokia SR Linux/SR OS,
MikroTik RouterOS 7, FortiOS, FRR, Aruba CX, Dell OS10, SONiC, Arcos, ...).
Output: a single JSON object describing hostname, enabled modules, interfaces
(name, address, type, OSPF/BGP attachment), and routing protocol state.
Results
Held-out set: 236 configurations from topologies never seen in training
(the split is by topology seed, not by sample, so no config from a training
topology appears in evaluation).
Both rows use the identical prompt, the identical Q4_K_M quantization, the
identical serving stack (LM Studio), and temperature=0. The only variable is
the adapter.
| Metric | Qwen3-8B baseline | + this adapter | delta |
|---|
| Field F1 | 0.5752 | 0.9219 | +0.3467 |
| JSON validity | 0.9492 | 1.0000 | +0.0508 |
| Exact match | 0.0000 | 0.1992 | +0.1992 |
| Hallucination rate | 0.0104 | 0.0073 | -0.0031 |
| Parseable outputs | 224 / 236 | 236 / 236 | +12 |
Precision 0.8975, recall 0.9610.
Without the schema in the prompt, the base model scores F1 0.0227 - it
cannot guess the target shape. The 0.5752 baseline above is the fair one: the
base model is given the full schema as a system prompt, the same one the adapter
was trained against.
General capability check
Fine-tuning a model on a narrow task often destroys its general ability. Measured
on 200 MMLU questions across 8 subjects:
| Baseline | + adapter |
|---|
| MMLU (200 q) | 0.630 | 0.640 |
The difference is 2 questions out of 200 (z = 0.21). The correct reading is
no measurable regression, not an improvement. 35% of the training mix was
general instruction data (FineTome-100k) specifically to protect against this.
Pre-registered thresholds
Thresholds were fixed in writing before training started:
| Threshold | Target | Result | |
|---|
| Field F1 | >= 0.85 | 0.9219 | pass |
| MMLU | >= 0.60 | 0.640 | pass |
| Hallucination | <= 0.005 | 0.0073 | missed |
Hallucination improved but did not reach the target. Reported as measured.
Known limitations
It fills in plausible defaults. The clearest failure mode: on an SR Linux
config with no MTU statement, the model emitted "mtu": 1500 on every interface.
1500 is the correct default for Ethernet - it is simply not in the source text.
This is the main source of the remaining precision gap.
It occasionally drops a module. On one EOS config the expected module list
was ["bfd", "bgp", "mpls"] and the model produced ["bfd", "bgp"], missing
MPLS, while getting every BGP neighbor, AS number, and community list exactly right.
Synthetic training data. All configurations were generated with
netlab - real, vendor-valid syntax produced by netlab's
own templates, but not drawn from production networks. Real-world configs carry
comments, dead blocks, local conventions, and vendor quirks that this set does
not represent. Expect degradation on production text.
Coverage is uneven. RouterOS 7 (400 samples) and EOS (367) are well
represented; SONiC (3) is not.
Training
| |
|---|
| Base | Qwen/Qwen3-8B |
| Method | LoRA, r=32, alpha=64, dropout=0.05 |
| Trainable | 87,293,952 params (1.05%) |
| Data | 2,541 task samples + 889 general (FineTome-100k, 35%) |
| Epochs | 2 (422 optimizer steps) |
| LR | 2e-4, cosine, 3% warmup |
| Batch | 1 x 16 gradient accumulation |
| Max length | 4096 |
| Precision | bf16 |
| Best eval loss | 0.0126 |
Hardware
Trained on an ASUS ProArt PX13 - AMD Ryzen AI Max+ 395 (Strix Halo, gfx1151),
Radeon 8060S, 128 GB unified memory - under WSL2 / Ubuntu 24.04 with
ROCm 7.2.1, librocdxg 1.2.0, and torch 2.9.1+rocm7.2.1.
About 72 s per optimizer step, 7.5 hours total.
A note for anyone attempting this: AMD's own ROCm documentation states
"No ML training support" for Ryzen APUs under native Windows, and native-Windows
training on gfx1151 did in fact fail here - every GPU-to-CPU scalar read returned a
stale host buffer, producing a silently frozen loss and a zero-valued saved adapter.
WSL2 works. An 11-test data-path integrity check (scalar reads, boolean masking,
masked loss vs CPU reference, forward, backward, optimizer step, weight readback)
passes under WSL2 and fails under native Windows. Run that check before trusting
any training run on this hardware.
Usage
For llama.cpp / LM Studio / Ollama, use the pre-quantized GGUF build:
huseyinturkdogan/netcfg-8b-GGUF
Append /no_think to the user message to suppress Qwen3's reasoning mode. All
numbers above were measured with it.
Dataset and evaluation code