LoRA adapter for entity-scoped classification of Brazilian regulatory filings, fine-tuned on Llama-4-Scout-17B-16E-Instruct via Adaption's AutoScientist platform.

Given a filing (a CVM Fato Relevante or Comunicado ao Mercado) and a named target entity, the adapter emits four independent labels — EVENT, STATUS, SCOPE, SIGNAL — each drawn from a fixed vocabulary. The SCOPE field is the point of the task: it forces the model to classify with respect to the tagged entity, distinguishing an event about the target from an event about another company named in the same document.

Task

This is closed-label classification, not open generation. Each field has a fixed set of values; anything outside them is a label hallucination.

Field	Values
EVENT	EARNINGS, DIVIDEND, BUYBACK, MERGER_ACQUISITION, CAPITAL_RAISE, DEBT, REGULATORY, LEGAL, OPERATIONAL, MANAGEMENT, RESTRUCTURING, OTHER
STATUS	CONFIRMED, COMPLETED, PENDING_APPROVAL, UNDER_NEGOTIATION, RUMOR, DENIED, CANCELLED
SCOPE	TARGET_ENTITY, OTHER_ENTITY, SECTOR, MARKET_WIDE
SIGNAL	POSITIVE, NEGATIVE, NEUTRAL, MIXED, UNCERTAIN

Output format is four lines, one value each, no explanation:

EVENT=<value>
STATUS=<value>
SCOPE=<value>
SIGNAL=<value>

Usage

This is a LoRA adapter. Load it on top of the base model with PEFT. The prompt at inference must match the training template — a differently-formatted prompt degrades results on the same task.

python

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "meta-llama/Llama-4-Scout-17B-16E-Instruct"
tokenizer = AutoTokenizer.from_pretrained(base)
base_model = AutoModelForCausalLM.from_pretrained(
    base, torch_dtype=torch.float16, device_map="auto",
)
model = PeftModel.from_pretrained(
    base_model, "Fernandosr85/fatobr-cvm-entity-scoped-adapter",
)

prompt = """Classify the following Brazilian regulatory filing with respect to the target entity below.
Target entity: BRF S.A.

Text:
A BRF S.A. ("BRF" ou "Companhia") comunica ao mercado que recebeu notificacao da
BlackRock, Inc. informando alteracao de participacao acionaria, reduzindo sua
posicao para aproximadamente 3,4% das acoes ordinarias de emissao da Companhia.

Answer with EVENT, STATUS, SCOPE, and SIGNAL, one value each, in this exact format:
EVENT=<value>
STATUS=<value>
SCOPE=<value>
SIGNAL=<value>"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

# Expected:
# EVENT=REGULATORY
# STATUS=CONFIRMED
# SCOPE=TARGET_ENTITY
# SIGNAL=NEUTRAL

Inference note. The 109B base does not fit on free-tier (T4/P100) GPUs; a larger-VRAM GPU is required to load it.

Evaluation results

Training win rates (Adaption internal)

Model	Win Rate (in-domain)	Win Rate (Corporate & Business)
Base (`Llama-4-Scout-17B-16E-Instruct`)	27%	43%
Adapted	73%	57%

A +46 point improvement in-domain. Unlike a typical task-specific adapter, the Corporate & Business reference category also improved (+14) rather than regressing — consistent with a closed-vocabulary task, where the win-rate judge has little room to reward style over correctness. No catastrophic forgetting was observed on the reference category.

Fase 1 diagnostic (simple prompt vs. scaffolded label)

There is no independent human-labeled reference in Portuguese for this task, so the table below is not an external-ground-truth baseline. It measures whether a simple prompt (label lists only, no per-value definitions) reproduces what a carefully-scaffolded prompt produces:

Field	Agreement, eval set	Agreement, OTHER_ENTITY slice (n=12)
EVENT	83.3%	90.9%
STATUS	76.2%	90.9%
SCOPE	95.2%	36.4%
SIGNAL	71.4%	45.5%

The eval-set SCOPE figure is dominated by TARGET_ENTITY rows and says little. On the genuine OTHER_ENTITY slice, SCOPE agreement drops to 36.4% — the unscaffolded model defaults to TARGET_ENTITY and attributes another party's event to the tagged entity roughly two times in three. Learning that entity-scope distinction is what this adapter is trained to do.

A holdout evaluation of this adapter against fatobr_sft_eval.jsonl (per-field accuracy, with the OTHER_ENTITY slice reported separately) is the comparable number and is pending; it requires loading the 109B base on adequate GPU. Fill this in before treating the adapter's accuracy as established.

Train/eval metrics

Metric	Value
Loss curve	decrease then plateau
Train vs validation	tracked closely, no overfit divergence
LR scheduler	cosine (warmup 0.05)
Gradient norm	initial spike, stable thereafter

Model details

Field	Value
Base model	`meta-llama/Llama-4-Scout-17B-16E-Instruct` (109B, 16-expert MoE)
Training method	Supervised Fine-Tuning (SFT) + LoRA
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0
Trainable modules	k/o/q/v_proj + shared_expert (gate/up/down) + feed_forward (gate/up/down)
Epochs	1
Batch size	max
Learning rate	5e-5 (cosine, 0.5 cycles)
Warmup ratio	0.05
Weight decay	0.05
Max grad norm	1.0
Train on inputs	false

Training dataset

Fernandosr85/adaption-brazilian-regulatory-filings

~290 prompt/completion pairs, built from the CVM IPE dataset with a deliberate methodology:

Step	Detail
Source year	2023 (2022+ avoids the legacy OLE2 format in older filings)
Categories	Fato Relevante and Comunicado ao Mercado only
Text extraction	Per-page; pages below a character floor (chart/table-only) dropped
Excluded	Investor decks / earnings calls (multi-event, don't map to one label set)
Identity	Anchored on CNPJ (stable across renames), name read from the filing text
Second entity	Filings naming a genuine second entity labeled once per entity, to exercise SCOPE; service providers (auditors, advisory banks, rating agencies, the exchange/regulator) filtered out
Completions	Closed-taxonomy, not platform-regenerated
Split	Grouped by filer CNPJ, stratified by SIGNAL, best of N folds

Note on corporate renames. The CVM index reports a company's current legal name for a given CNPJ, even on documents filed years before a rename. Five real cases in this corpus (3R→Brava Energia, CTEEP→ISA Energia Brasil, Aliansce Sonae→Allos, Omega→Serena Energia, Via→Grupo Casas Bahia) required anchoring identity on CNPJ and reading the name from the filing text rather than trusting the index name.

Known limitations

Completion regeneration. Adaptive Data regenerates completions on every run; a freeform Blueprint cannot enforce membership in a closed label set. This adapter was trained on the original (verbatim) completions, not the platform-enhanced ones, which invented values outside the taxonomy (observed EVENT/SCOPE fabrications such as Ownership Change, Divestiture, Entity-Specific in the Enhanced column). Downstream users of the platform on closed-label tasks should map the original completion column.
Pilot scale. ~290 filings; the genuine OTHER_ENTITY slice is small (~12 rows), so every figure on that axis is directional, not conclusive.
Model-generated reference labels. The training labels are produced by a language model against the closed taxonomy, traceable to a per-row justification, but not human-adjudicated.
Holdout accuracy pending. The Adaption win rate is the platform's internal metric; a per-field holdout evaluation of this adapter is not yet run (see the diagnostic note above).

Model repositories

Platform	Link
Kaggle Model	fernandosr85/fatobr-evidence-guard-adapter
HuggingFace Dataset	Fernandosr85/adaption-brazilian-regulatory-filings
Kaggle Diagnostic	fatobr-cvm-entity-scoped-filings

Credits

Fine-tuning platform: Adaption — AutoScientist & Adaptive Data
Challenge: AutoScientist Challenge 2026 — Market-Analysis & News category
Data source: CVM Dados Abertos — Cias Abertas: Documentos Periódicos e Eventuais (IPE) (public)
Base model: meta-llama/Llama-4-Scout-17B-16E-Instruct (Llama 4 Community License)
Author: Fernando Rodrigues · Kaggle: fernandosr85 · HuggingFace: Fernandosr85

Disclaimer

Experimental research artifact submitted to the AutoScientist Challenge 2026 (Market-Analysis & News category). Classification outputs are automated and require review before any operational use. Derived from public CVM data; the exact terms-of-use text was not independently confirmed at the time of writing and should be verified before redistribution.