You can deploy this model and start earning money today!

Mr-Rosen/
Accuracy-Is-Not-Enough-FinQA-LoRA

Name: Accuracy-Is-Not-Enough-FinQA-LoRA
Availability: InStock

Mr-Rosen/Accuracy-Is-Not-Enough-FinQA-Dataset

Qwen/Qwen2.5-7B-Instruct

adapter

mit

Views

No views yet

FinQA Qwen2.5-7B LoRA

A LoRA adapter for structured financial question answering over Natively Extended FinQA.

Base model:

Qwen/Qwen2.5-7B-Instruct

This repository contains PEFT adapter weights, not a standalone copy of the base model.

Official Result

Metric	Final Test
Execution Accuracy	66.17%
Program Accuracy	61.64%
Parse Success	97.82%
Average Latency	0.4793 s/example

Selected checkpoint:

epoch_1_adapter

The checkpoint was selected using development-set performance before final test evaluation.

Intended Input

This adapter was trained on full expanded FinQA context.

Training mapping:

text

question
+ pre_text
+ table
+ post_text
        ↓
FinQA program

Training target:

qa.program

The selected prompt is included in this repository as:

S2_financial_analyst_operation_reader.json

Do not replace the full-document input with RAG chunks when attempting to reproduce the reported LoRA result.

Adapter Configuration

text

rank:        64
alpha:       32
dropout:     0.05
bias:        none
task type:   CAUSAL_LM

Target modules:

text

q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj

Additional confirmed training settings include:

text

learning rate:         1e-4
effective batch size:  32
training dtype:        bfloat16

Official Training Burden

For the study's controlled practical comparison:

text

Training time: 14.7519 hours
Training cost: approximately $27.88

Installation

pip install torch transformers peft accelerate safetensors

Load the Adapter

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-7B-Instruct"
ADAPTER = "Mr-Rosen/Accuracy-Is-Not-Enough-FinQA-LoRA"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER,
)

model.eval()

Prepare the Input

Use the exact released S2_financial_analyst_operation_reader prompt.

The prompt should receive:

text

qa.question
pre_text
table
post_text

Do not expose:

text

qa.program
qa.exe_ans
qa.gold_inds

during normal dev or test inference.

Generate

Once rendered_prompt has been constructed with the official prompt:

python

inputs = tokenizer(
    rendered_prompt,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
    )

generated = output[0, inputs["input_ids"].shape[-1]:]

text = tokenizer.decode(
    generated,
    skip_special_tokens=True,
)

print(text)

Expected Output

["subtract(", "5829", "5735", ")", "EOF"]

Evaluation

Parsed predictions should be converted to:

json

[
  {
    "id": "example-id",
    "predicted": [
      "subtract(",
      "5829",
      "5735",
      ")",
      "EOF"
    ]
  }
]

Then evaluate them with the original FinQA evaluator.

Related Repositories

Dataset:

Mr-Rosen/Accuracy-Is-Not-Enough-FinQA-Dataset

Prompts, evaluator, results, and paper materials:

MarkPaulRosenthal/Accuracy-Is-Not-Enough-Practical-Financial-QA

Limitations

This adapter was trained specifically for FinQA-style numerical program generation.

It is not a general financial-advice model, and its reported accuracy should not be assumed to transfer directly to unrelated financial documents or tasks.

License

The adapter is released under the MIT License.

The Qwen base model remains subject to its own license.

The FinQA-derived training dataset is separately released under CC BY 4.0.