A Qwen-1.5B LoRA that extracts argument structure — claims, premises, citations, and support/attack relations — from political claims and argumentative prose. This is the fourth iteration of a distillation project, and the one that actually works well enough to plug into a real fact-checking pipeline.

Where this came from

I built an argument-aware retrieval pipeline for Politifact-style fact-checking. Given a claim like "Politician X said Y," it retrieves evidence using argument-role-targeted queries and returns a 6-way truth verdict. When the parser was reading gold argument annotations from the LIARArg dataset, this pipeline hit 0.422 6-way F1 versus 0.114 for a flat-RAG baseline. The 0.308 gap was the whole reason to build it.

But that gap only means something if a real parser can slot in at inference time. Gold annotations aren't available for actual incoming claims. So the question became: what parser closes that gap?

First cut, Phase 2-α, was gpt-oss-120b running zero-shot on Cerebras. It closed about 45% of the gap (integration F1 0.254). Real, useful, but calling a 120B cloud model per claim isn't practical for anything resembling deployment.

Phase 2-β was the distillation project. Four iterations. Can a small local model preserve most of the gain?

The four iterations

v1 was Qwen-0.5B, full fine-tune on four argument-mining corpora (AbstRCT, Microtext, CDCP, PERSPECTRUM), 1,494 records, 3 epochs. The smallest reasonable baseline. In-domain comp-F1 averaged 0.108 across the four held-out test sets. High empty rates on some domains (PERSPECTRUM: 91% empty). The point was to get the pipeline plumbing right, not to publish a number.

v2 kept the same data but moved to Qwen-1.5B with LoRA r=16 (α=32, dropout 0.05, target q_proj,k_proj,v_proj,o_proj). 3 epochs. In-domain comp-F1: 0.219, roughly double v1. Microtext premise F1 jumped from 0.000 to 0.680. AbstRCT empty rate 75% → 50%. Scale + LoRA + longer training context are the dominant levers here.

v3 was v2's adapter continued for one more epoch after adding a fifth corpus (AAEC, 402 persuasive essays). Marginal in-domain improvement (0.229). PERSPECTRUM actually regressed slightly, which was the first sign that adding more of the same kind of extractive gold hits diminishing returns quickly. Then I tried v3 on the actual LIARArg parse and hit 83% empty rate on the first 64 rows. Killed that run. The lesson was clear: extractive gold from academic argument-mining corpora doesn't teach a small student to handle Politifact-style claims. The distribution gap is too wide.

v4 is this model. Two changes from v3:

Fresh adapter, not continual. Clean A/B against v3.
I generated 2,123 silver labels on LIARArg train articles using gpt-oss-120b via Cerebras. Both the extracted argument structure and the model's Chain-of-Thought reasoning came back (the CoT was in message.reasoning, which I captured almost by accident). The training code's target-formatter passes reasoning through as <think>...</think>{json} when present, so v4 accidentally became CoT-aware for LIARArg-style inputs — while staying purely extractive on the gold in-domain corpora (where reasoning was empty).

This wasn't planned. It just happened, and it turned out to be exactly what was missing.

Training details: 3,617 records total, 3 epochs, fresh LoRA adapter, fp16, Adafactor, gradient checkpointing. Batch 1, grad accum 32. About 29 hours on a single GTX 1080 Ti.

What v4 actually does

The load-bearing number is Phase 1 integration on LIARArg — the whole reason to build any of this:

Metric	v4	Phase 2-α teacher (120B)	Teacher retention
6-way F1	0.217	0.254	85%
3-way F1	0.457	0.461	99%
within-1 accuracy	0.605	0.616	98%

Flat-RAG baseline for reference: 0.114. v4 beats it by 0.103 and closes 33% of the gold-parser gap using a locally-runnable 1.5B model.

LIARArg parse empty rate went from v3's 83% down to 23%. Silver + CoT was the missing piece.

In-domain retention is modest — 0.192 comp-F1 averaged across the five training corpora, slightly regressed from v3's 0.229. That's the cost of a fresh 3-epoch run versus v3's effective 4 epochs. The trade was worth it for the cross-domain transfer.

OOD probes

To characterize where v4 stops transferring, I ran three unseen-domain probes:

AMPERSAND (Chakrabarty et al. 2019, Reddit ChangeMyView). Binary is-argumentative F1 = 0.819 with recall 0.970 on 150 balanced sentences. v4 catches essentially all argumentative content in Reddit debate. The 30% false-positive rate comes from over-flagging fragments and borderline sentences; some of those are arguably right and just disagree with the annotator.

PERSUADE 2.0 (Kaggle Feedback Prize, student argumentative essays). Component F1 macro = 0.193 with 45.7% extraction rate on 25 essays. Claim F1 = 0.351, premise F1 = 0.034. v4 finds about half of PERSUADE's argumentative spans and labels claim-like content reasonably. Premise F1 collapses because PERSUADE's Evidence and Rebuttal categories are much narrower than v4's premise concept.

ECHR (European Court of Human Rights case briefs). Component F1 = 0.074 with 9.7% extraction rate against a proxy gold derived from ECHR's agent labels. Legal reasoning is structurally distant from anything v4 saw in training.

Ordering (Reddit > essays > legal) tracks discourse-register proximity to v4's training data. Predictable, but useful to have quantified.

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch, re, json

base_id    = "Qwen/Qwen2.5-1.5B-Instruct"
adapter_id = "iamjayeshc/ArgParser-v4-Qwen1.5B"

tok = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.float16, device_map="auto"
)
model = PeftModel.from_pretrained(base, adapter_id)

INSTR = ("Extract all argument components and relations from the text. "
         "Output strict JSON with claim_components, premise_components, "
         "citation_components, and relations.")

def parse(text, max_new_tokens=1024):
    prompt = tok.apply_chat_template(
        [{"role": "user", "content": f"{INSTR}\n\nTEXT:\n{text}"}],
        tokenize=False, add_generation_prompt=True,
    )
    enc = tok(prompt, return_tensors="pt", truncation=True,
              max_length=2048).to(model.device)
    out = model.generate(**enc, max_new_tokens=max_new_tokens,
                         do_sample=False,
                         pad_token_id=tok.pad_token_id,
                         eos_token_id=tok.eos_token_id)
    raw = tok.decode(out[0, enc["input_ids"].shape[-1]:],
                     skip_special_tokens=True)
    cleaned = re.sub(r"<think>.*?</think>", "", raw, flags=re.DOTALL)
    m = re.search(r"\{.*\}", cleaned, flags=re.DOTALL)
    return (json.loads(m.group()) if m else None), raw

pred, raw = parse(
    "The Obama administration is putting Border Patrol agents in a chokehold."
)
print(pred)

Prompt:

Extract all argument components and relations from the text.

Return strict JSON with:

claim_components
premise_components
citation_components
relations

Output

{ "claim_components": [], "premise_components": [], "citation_components": [], "relations": [] }

Things worth knowing before using this

The model emits <think>...</think> blocks on political and opinionated inputs (that's where it saw CoT during training). On formal or structured text — legal writing, some scientific abstracts — it goes straight to JSON. Neither is a bug, it just needs handling if you're stripping the raw generation.

Long inputs can trigger a relation-generation loop that leaves the JSON unclosed. What happens: the model emits valid claim_components and premise_components early, then gets stuck emitting {"src": N, "tgt": M, "type": "support"} tuples in a repeating pattern until it hits max_new_tokens. You never see the closing } so strict JSON parsing rejects everything. The workaround is either setting max_new_tokens conservatively, or using a lenient parser that pulls each valid {...} object out of each section independently and dedupes relations. I use the second approach in the training repo.

v4 systematically over-predicts spans on OOD text — precision runs lower than recall in every probe. On borderline sentences it defaults to labeling as claim. Something to keep in mind if downstream consumers care about precision.

Fine-grained annotation schemas map imperfectly to v4's binary claim/premise split. PERSUADE's Evidence/Rebuttal distinction and ECHR's Court/Applicant/State agent labels don't translate directly. v4 knows a claim from a premise; it doesn't know PERSUADE's Evidence from PERSUADE's Rebuttal.

Training summary

Base: Qwen/Qwen2.5-1.5B-Instruct
LoRA: r=16, α=32, dropout 0.05, on q_proj,k_proj,v_proj,o_proj
Training records: 3,617 (5 gold corpora + 2,123 LIARArg silver)
Epochs: 3, fresh adapter
Optimizer: Adafactor, fp16, gradient checkpointing
Hardware: single NVIDIA GTX 1080 Ti
Wall clock: ~29 h

Acknowledgements

This repository is maintained by Jayesh Choudhari.

The ArgParser research project was developed collaboratively, and this repository mirrors the final model with permission from the original collaborator.

Citation

A formal preprint is planned. Until then, please cite this Hugging Face repository if you use the model in your work.

License

Apache 2.0. Base model (Qwen 2.5) is also Apache 2.0. Use however you want, no warranty.