from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch

model = AutoModelForSeq2SeqLM.from_pretrained(
    "Corp-o-Rate-Community/statement-extractor",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "Corp-o-Rate-Community/statement-extractor",
    trust_remote_code=True,
)

text = "Apple Inc. announced a commitment to carbon neutrality by 2030."
inputs = tokenizer(f"<page>{text}</page>", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=2048, num_beams=4)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

Input Format

Wrap your text in <page> tags:

<page>Your text here...</page>

Output Format

The model outputs XML with extracted statements:

xml

<statements>
  <stmt>
    <subject type="ORG">Apple Inc.</subject>
    <object type="EVENT">carbon neutrality by 2030</object>
    <predicate>committed to</predicate>
    <text>Apple Inc. committed to achieving carbon neutrality by 2030.</text>
  </stmt>
</statements>

Entity Types

Type	Description
ORG	Organizations (companies, agencies)
PERSON	People (names, titles)
GPE	Geopolitical entities (countries, cities)
LOC	Locations (mountains, rivers)
PRODUCT	Products (devices, services)
EVENT	Events (announcements, meetings)
WORK_OF_ART	Creative works (reports, books)
LAW	Legal documents
DATE	Dates and time periods
MONEY	Monetary values
PERCENT	Percentages
QUANTITY	Quantities and measurements

Demo

Try the interactive demo at statement-extractor.vercel.app

Training

Base model: google/t5gemma-2-270m-270m
Training examples: 77,515
Final eval loss: 0.209
Training with refinement phase (LR=1e-6, epochs=0.2)
Beam search: num_beams=4

About corp-o-rate

This model is part of corp-o-rate.com - an AI-powered platform for ESG analysis and corporate accountability.

License

MIT License