A LoRA adapter for Qwen/Qwen3-32B, fine-tuned on CUAD (Contract Understanding Atticus Dataset) clause Q&A pairs. Built end-to-end on a single NVIDIA Grace Blackwell GB10 (DGX Spark, 128 GB unified memory) with the same NF4 train, NVFP4 serve, EAGLE-3 speculative decoding stack used across the Pylox Forge portfolio.

This is the 32B-tier counterpart to the 8B legal-cuad adapter, intended for harder clause interpretation where 8B reasoning fails. Same training corpus, larger model. Honest framing on what changed and what did not is in the Evaluation section.

Model details

Adapter: LoRA (PEFT, rank 32, alpha 64, dropout 0.1)
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Base model: Qwen/Qwen3-32B
Recommended speculative head: RedHatAI/Qwen3-32B-speculator.eagle3
License: Apache 2.0 (inherited from Qwen3-32B)
Hardware: NVIDIA Grace Blackwell GB10 (DGX Spark, 128 GB UMA)

Training data and technique

Source: theatticusproject/cuad-qa (Contract Understanding Atticus Dataset)
Examples after dedup, quality filter, and PII redaction: 331
Format: Standard chat messages format with assistant-only loss (TRL 1.2)
Method: NF4 QLoRA SFT (4-bit NormalFloat base, bfloat16 compute, double quantization)
Hyperparameters: 3 epochs, cosine LR (2e-4 peak), max_seq_length 2048, sequence packing enabled, optim: adamw_torch_fused (Qwen3 thinking mode disabled), gradient accumulation 16
Notes: low_cpu_mem_usage=True was required to prevent double-buffering OOM during 32B weight loading on the unified memory architecture.

Evaluation

Numbers are pulled directly from the local benchmark JSON. No invented values.

Throughput on Grace Blackwell GB10

Metric	Value
Sustained throughput	15.4 tok/s
Concurrent batch-8 throughput	99.1 tok/s
TTFT p50	322.5 ms
TTFT p95	340.9 ms
End-to-end latency p50	9691 ms

Cost (NVFP4 serving)

Metric	Value
Cost per 1M output tokens	$0.9019
Comparable OpenAI GPT-4o output	$10 per 1M
Savings factor	11.1x cheaper than GPT-4o

Capability preservation (academic, lm-evaluation-harness)

lm-eval failed (rc=1) on this run: likely a task-name compatibility issue with the current lm-eval version against Qwen3-32B's tokenizer. Re-run with the academic harness fix is in progress; numbers will be added when available.

Domain (LegalBench subset)

LegalBench legalbench_subset reported "no supported tasks" for the configured task list. A LegalBench task list compatible with Qwen3-32B's chat template needs to be configured before legal-domain numbers can be reported. This is the same finding as on the 8B legal adapter and is a known evaluation gap, not a model gap.

Safety (with Pylox safety gateway, 50-prompt red team)

Metric	Value
Adversarial block rate	77.78%
False positive rate on benign controls	0.0%

Zero false positives on benign controls is the right safety profile for a legal-domain adapter.

Why use the 32B over the 8B legal adapter

The 32B tier is meaningful when:

The 8B adapter misclassifies a clause and you need a second opinion with more reasoning capacity
The contract uses unusual or novel clause language outside CUAD's taxonomy
Single-user latency in the 5 to 10 second range is acceptable (this 32B at NVFP4 sits at ~9.7s p50 end-to-end)

The 8B tier is correct when:

Latency must stay under 4 seconds (the 8B sits at 3.7s p50 end-to-end)
The clause is in CUAD's standard taxonomy
Cost per query matters more than reasoning depth

Quickstart

PEFT (research / batch)

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_id = "Qwen/Qwen3-32B"
adapter_id = "pyloxsystems/legal-qwen3-32b-lora"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    low_cpu_mem_usage=True,
)
model = PeftModel.from_pretrained(model, adapter_id)

messages = [
    {"role": "system", "content": "You are a contract review assistant. Cite the clause and explain plainly."},
    {"role": "user", "content": "Does this MSA contain an unusual indemnification carve-out for IP infringement?\n\n[paste contract text]"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0], skip_special_tokens=True))

vLLM with EAGLE-3 speculative decoding

bash

vllm serve Qwen/Qwen3-32B \
    --enable-lora \
    --lora-modules legal-qwen3=pyloxsystems/legal-qwen3-32b-lora \
    --speculative-config '{
        "method": "eagle3",
        "model": "RedHatAI/Qwen3-32B-speculator.eagle3",
        "num_speculative_tokens": 5
    }'

Intended use

Second-opinion clause analysis when the 8B legal adapter misclassifies
Interpretive analysis on unusual or novel commercial contract clauses
Drafts of analysis on complex contracts (NOT final legal opinion)
Pipeline demonstration of the Pylox Forge stack on the 32B tier

Out of scope

Final legal advice. All outputs require licensed-attorney review before any decision.
Real-time interactive legal chat (32B latency is roughly 3x the 8B at single-user)
Documents in languages other than English
Production legal review without a substantially larger fine-tune corpus and rigorous LegalBench / CUAD F1 evaluation

Limitations

Same training corpus as the 8B adapter (331 examples). The 32B tier is a capacity upgrade, not a data upgrade.
Higher latency than the 8B tier. Choose 8B when response time matters more than reasoning depth.
Academic and domain benchmarks are gaps; numbers will be added as evaluations complete.
Qwen3 thinking mode is disabled in this configuration (assistant_only loss). Re-enabling thinking mode requires a re-train.

License

Inherits Apache 2.0 from the Qwen3-32B base model.

Citation

bibtex

@misc{pylox_legal_qwen3_32b_2026,
  author       = {Girard, Emilio},
  title        = {Pylox Legal Premium 32B (legal-qwen3-32b)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/pyloxsystems/legal-qwen3-32b-lora}}
}

Pylox Forge is a solo-operated LLM fine-tuning lab on NVIDIA Grace Blackwell. Site: pyloxforge.com. Other adapters: pyloxsystems on Hugging Face.