A LoRA-adapted Gemma 4 31B model fine-tuned for multi-step US Congressional bill analysis — built end-to-end using the Adaption Labs AutoScientist platform for the 2026 Adaption AutoScientist Challenge (Legal Category).
Model Details
Model Description
LegalMind-Gemma4-31B-BillAnalyst is a parameter-efficient LoRA adapter that transforms Google's Gemma 4 31B instruction-tuned model into a specialized legal reasoning engine for US Congressional legislation. Unlike simple bill summarization models, it performs 8 distinct multi-step legal analysis tasks — from statutory interpretation with step-by-step reasoning traces to structured compliance checklist generation and explicit abstention when bill text lacks requested information.
The model was trained entirely through the Adaption Labs AutoScientist pipeline, which automated hyperparameter optimization, data enhancement, and the training loop. The result is an 82% training win rate against the unmodified baseline (up from just 18%), representing a 355% relative improvement.
Key observation: Train loss and eval loss converge closely throughout training (final gap: 0.011), confirming no overfitting. Eval loss monotonically decreased from 1.049 → 0.751 across all 5 checkpoints.
Uses
Direct Use
This model is designed for automated legal analysis of US Congressional legislation. It can be used directly for:
Statutory interpretation — Analyzing the legislative mechanism, instruments used, and enforcement pathways of a bill
Impact analysis — Assessing stakeholders, fiscal implications, and unintended consequences
Compliance checklist generation — Extracting structured obligations, timelines, and penalties into table format
Amendment analysis — Identifying and analyzing existing laws being modified by a bill
Comparative bill analysis — Side-by-side comparison of two competing legislative approaches
Key provisions extraction — Parsing provisions into structured tables with type classification (Obligation/Authorization/Definition)
Abstention — Correctly refusing to fabricate information when the bill text doesn't contain the requested data
Downstream Use
Legal technology platforms for automated legislative monitoring
Congressional research assistants for policy analysts
Regulatory compliance workflows for organizations tracking federal legislation
Legal education tools for law students studying statutory interpretation
Government affairs teams analyzing pending legislation
Out-of-Scope Use
Not legal advice: This model is a research and analysis tool. Its outputs should never be treated as professional legal counsel or relied upon for legal decisions.
Non-US legislation: Trained exclusively on US Congressional bills; not designed for UK, EU, or other jurisdictions.
Case law analysis: Not trained on judicial opinions, court rulings, or case precedents.
Contract review: Not designed for contract clause analysis or commercial agreement review.
Real-time legal research: Does not have access to current legislation or legal databases.
Bias, Risks, and Limitations
Jurisdictional bias: Trained exclusively on US federal legislation. Performance on state-level bills, foreign legislation, or international law is untested and likely degraded.
Temporal bias: Training data consists of historical Congressional bills. The model may not reflect current legislative drafting conventions or recently enacted legal frameworks.
Hallucination risk: While the model includes abstention training (~11% of dataset), it may still occasionally fabricate statutory references, dollar amounts, or section numbers not present in the input text.
Base model limitations: Inherits the biases, factual limitations, and failure modes of the underlying Gemma 4 31B architecture.
Structured output consistency: The model is trained to produce tables, checklists, and structured formats, but may occasionally produce malformed markdown or inconsistent formatting.
Training data scope: The 6,648 training examples are derived from 1,000 unique bills. Analysis patterns may be less robust for bill topics underrepresented in the BillSum sample.
Recommendations
Always verify model outputs against the original bill text before use in any professional context.
Use the model's reasoning traces (<reasoning_start> / </reasoning_start>) to audit the analytical process and identify potential errors.
Cross-reference extracted dollar amounts, section numbers, and dates with the source document.
Treat the model as an analytical assistant, not an authoritative legal source.
Be especially cautious with abstention — if the model says information is "Not specified in bill text," verify this claim manually.
How to Get Started with the Model
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
34# Load base model5base_model = AutoModelForCausalLM.from_pretrained(6"google/gemma-4-31B-it",7 torch_dtype="auto",8 device_map="auto"9)10tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-31B-it")1112# Load LoRA adapter13model = PeftModel.from_pretrained(14 base_model,15"lohithaaa/adaption-us-bill-analysis-tasks"16)1718# Example: Statutory Interpretation19prompt ="""Analyze the following US Congressional bill and identify its primary
20legislative mechanism. Explain step-by-step: (1) what existing law or framework
21it modifies or creates, (2) the specific legal instruments used (amendments, new
22sections, appropriations, mandates), and (3) the enforcement or implementation
23pathway.
2425SECTION 1. SHORT TITLE.
26 This Act may be cited as the 'Fair Lending for All Act'.
2728SEC. 2. PROHIBITION ON DISCRIMINATORY LENDING.
29 (a) IN GENERAL.—No covered lender shall deny or vary the terms of a
30 residential mortgage loan based on the race, color, religion, national
31 origin, sex, or age of the applicant.
32 (b) ENFORCEMENT.—The Bureau of Consumer Financial Protection shall
33 enforce this section under the authorities granted by the Consumer
34 Financial Protection Act of 2010.
35 (c) PENALTIES.—Any lender found in violation of subsection (a) shall
36 be subject to a civil penalty of not more than $100,000 per violation."""3738inputs = tokenizer(prompt, return_tensors="pt").to(model.device)39outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.7)40print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Source: 1,000 US Congressional bills from the BillSum dataset (CC0 Public Domain), programmatically transformed into 6,648 multi-task legal reasoning examples.
Task Distribution:
Task Type
Count
%
What It Teaches
Statutory Interpretation
1,000
15.0%
Legislative mechanism analysis with reasoning traces
Impact Analysis
1,000
15.0%
Multi-factor stakeholder and fiscal assessment
Compliance Checklist
1,000
15.0%
Structured obligation extraction into tables
Risk Assessment
1,000
15.0%
Constitutional & litigation risk scoring
Key Provisions
1,000
15.0%
Table-format provision extraction with type classification
Abstention/Safety
723
10.9%
Explicit refusal when information is missing
Amendment Analysis
675
10.2%
Cross-referencing existing laws being modified
Comparative Analysis
250
3.8%
Side-by-side bill comparison
Key dataset design decisions:
Every output includes <reasoning_start> / </reasoning_start> step-by-step reasoning traces
Deterministic metadata (dollar amounts, section counts, "shall"/"must" frequency) extracted directly from bill text
SHA-256 hash-based deduplication on output text
~11% abstention tasks teaching the model when NOT to answer
Capped at 6,648 rows (within the empirically optimal 5,000–8,500 range)
Training Procedure
Adaption Labs AutoScientist Pipeline
This model was trained entirely through the Adaption Labs AutoScientist platform, which automated data enhancement, hyperparameter search, and the training loop. Below are the exact pipeline configurations used:
Enhancement Recipe Configuration:
Recipe Toggle
Setting
Strategic Rationale
Prompt Rephrase
❌ OFF
Deliberately disabled to preserve deterministic numeric data. Our dataset contains exact dollar amounts ($100,000), section references (SEC. 2), and word-frequency counts ('shall' appears 14 times). Automatic rephrasing would corrupt these strict numeric constants into vague approximations, degrading training signal quality.
Prompt Deduplication
✅ ON
Second-layer deduplication on top of our pre-processing SHA-256 hash dedup, ensuring maximum dataset cleanliness and preventing the model from memorizing repeated output patterns.
Prompt Metadata Injection
✅ ON
Enriches each prompt with additional contextual metadata, improving the model's understanding of legal domain nuances and bill structure.
House Special
✅ ON
Adaption's most powerful proprietary combination of optimization recipes, applied for maximum quality uplift across all training examples.
Reasoning Traces
✅ ON
Critical toggle for our dataset — every output contains <reasoning_start> step-by-step deductive logic. This ensures the training loop specifically optimizes for interpretable, traceable reasoning pathways.
Hallucination Mitigation
✅ ON
Reinforces the ~11% abstention tasks where the model must refuse to fabricate information. Works in concert with our "Not specified in bill text" training examples to minimize hallucination of legal facts.
Blueprint (System Preamble):
A custom system preamble was configured in the Blueprint stage to align the enhancement pipeline with the dataset's behavioral objectives:
You are an expert legal analyst specializing in US Congressional legislation.
Rules:
1. Always use <reasoning_start> and </reasoning_start> tags for step-by-step analysis
2. Present outputs in structured formats (tables, numbered steps, checklists)
3. Reproduce numeric data exactly as stated — never approximate
4. If information is missing from bill text, state "Not specified in bill text"
5. Systematically evaluate: legislative mechanism, stakeholders, fiscal impact,
implementation challenges, and constitutional considerations
6. Never fabricate case citations, statutory references, or legal advice
This ensures the platform's enhancement algorithms optimize in the same direction as the training data — reinforcing reasoning traces, structured output, deterministic numeric accuracy, and explicit abstention behavior.
Training Hyperparameters
All hyperparameters were automatically selected by the Adaption Labs AutoScientist — no manual tuning was performed:
Training regime: bf16 mixed precision
Training method: Supervised Fine-Tuning (SFT)
Adapter: LoRA (Low-Rank Adaptation)
LoRA rank (r): 8
LoRA alpha: 8
LoRA dropout: 0.0
Target modules:q_proj, v_proj
Learning rate: 1e-4
LR scheduler: Cosine (0.5 cycles)
Min LR ratio: 0.1
Warmup ratio: 0.1
Weight decay: 0.0
Max gradient norm: 2.0
Epochs: 1
Batch size: 1 (max for memory)
Total training steps: 58
Train on inputs: false (output tokens only)
Number of evaluations: 5
Speeds, Sizes, Times
Total training steps: 58
Total floating point operations: 3.96 × 10¹⁸
Eval runtime per checkpoint: ~17.2 seconds
Eval throughput: 0.87 samples/second
Adapter size: 41.8 MB (adapter_model.safetensors)
Framework: PEFT 0.15.1, Transformers 5.10.1
Evaluation
Testing Data, Factors & Metrics
Testing Data
Evaluation was performed using the Adaption Labs platform's internal held-out evaluation methodology. The platform generates test prompts from the legal domain and uses an LLM-as-a-judge framework to compare the adapted model's responses head-to-head against the unmodified baseline.
Additionally, 5 validation checkpoints were evaluated during training using a held-out split from the training data (15 samples, 2 eval steps per checkpoint).
Reasoning quality: Assessed via step-by-step reasoning trace coherence
Numeric accuracy: Precision of extracted dollar amounts, section references, and dates
Abstention behavior: Correct refusal rate when bill text lacks requested information
Structured output quality: Consistency and completeness of tables, checklists, and formatted outputs
Metrics
Win Rate (primary): Percentage of head-to-head comparisons where the adapted model's response is judged superior to the base model's response by an LLM judge
Training Loss: Cross-entropy loss on training examples
Eval Loss: Cross-entropy loss on held-out validation split
Results
Metric
Value
Training Win Rate (Adapted)
82%
Training Win Rate (Base)
18%
Win Rate Delta
+64 points (355% relative)
Final Training Loss
0.761
Final Eval Loss
0.751
Train/Eval Loss Gap
0.011 (no overfitting)
Eval loss progression across 5 checkpoints:
Checkpoint
Step
Epoch
Eval Loss
1
14
0.24
1.0489
2
25
0.43
0.8542
3
36
0.62
0.7780
4
47
0.81
0.7589
5
58
1.00
0.7507
Eval loss decreased monotonically across all checkpoints, confirming stable convergence without overfitting.
Summary
The model achieves an 82% training win rate against the unmodified Gemma 4 31B baseline, a massive improvement from the 18% base performance. The 64-point gap demonstrates that the multi-task legal reasoning dataset successfully exposed and addressed fundamental weaknesses in the base model's ability to perform structured legal analysis, step-by-step reasoning, and principled abstention.
This result validates the data-centric approach: rather than scaling model parameters, switching to complex reasoning tasks that exploit the baseline deficit yielded dramatically superior results compared to simple summarization (which achieved only a ~4% gap in prior experiments).
LoRA (Low-Rank Adaptation): A parameter-efficient fine-tuning method that injects trainable low-rank matrices into specific layers of a pre-trained model, enabling adaptation without modifying the full parameter set.
SFT (Supervised Fine-Tuning): Training a model on input-output pairs where the correct output is provided.
Win Rate: The percentage of head-to-head comparisons where the adapted model's response is judged better than the base model's response by an LLM evaluator.
AutoScientist: Adaption Labs' automated ML pipeline that co-optimizes data quality and training hyperparameters.
