Component of CS-552 Spring 2026 multi-agent legal QA pipeline (Clanker-Scientists team). The reasoning module: given a contract excerpt + yes/no question, produces a reasoning trace followed by Yes/No.

This repo contains the LoRA adapter only. Use it on top of Qwen/Qwen3-14B.

Training

Base: Qwen/Qwen3-14B
Method: QLoRA (rank 32, alpha 64) on chain-of-thought self-distillation
- Phase A: generate 1,000 reasoning traces with base Qwen3-14B (thinking mode, max_new_tokens=512)
- Phase B: filter to 752 traces where the parsed answer matched the gold label
- Phase C: train adapter for 3 epochs at LR 1e-5 on (prompt, reasoning + answer)
- Phase D: evaluate with enable_thinking=True (apples-to-apples vs Phase 1 baselines)

Evaluation

LegalBench, N=25 per task, seed=0:

Task	Best baseline	v3 FT	Δ
`contract_nli_explicit_identification`	0.840 (Qwen2.5-7B think)	0.720	-0.120
`consumer_contracts_qa`	0.960 (Qwen3-14B think)	0.960	0.000 (matches)
`proa`	1.000 (Qwen3-14B no-think)	0.960	-0.040

CoT distillation provides selective benefit by task type: it lifts free-form QA tasks (consumer_contracts_qa) to baseline parity, slightly regresses structurally-fixed entailment tasks (contract_nli), and preserves near-ceiling performance on proa.

Usage

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

BASE = "Qwen/Qwen3-14B"
ADAPTER = "cs-552-2026-Clanker-Scientists/coordinator-qwen3-14b-qlora-legal-v3"

tok = AutoTokenizer.from_pretrained(BASE)
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)
base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, ADAPTER)
model.eval()

msgs = [
    {"role": "system", "content": "You are a legal contract analyst..."},
    {"role": "user", "content": "Contract:\n[excerpt]\n\nQuestion: Does the contract explicitly identify confidential information?"},
]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))