BERTimbau-large-metadata-council-pt is a baseline implementation, consisting of a fine-tuned BERT model for Named Entity Recognition (NER), to automatically extract metadata such as meeting number, date, location, participants, and time expressions from Portuguese municipal meeting minutes.
It was developed as part of a study on information extraction and indexing of administrative documents.

Key Features

🏛️ Domain-Specific: Trained exclusively on authentic municipal meeting minutes
🧠 Entity-Level Extraction: Identifies key metadata (minute ID, date, location, start/end times, participants)
⚙️ Transformer-based Architecture: BERTimbau backbone with fine-tuning for token classification
📈 Strong NER Performance: Achieves F1-score above 0.95 on the Portuguese dataset

Model Details

Base Model: neuralmind/bert-large-portuguese-cased
Architecture: BERT for token classification (NER)
Parameters: ~335M
- 24 transformer layers
- 1024 hidden dimensions
- 16 attention heads
- 4096 intermediate size
Max Sequence Length: 512 tokens
Learning Rate: 5e-5
Warmup: 0.1
Batch Size: 16
Optimizer: AdamW
Weight Decay: 0.01
Entity Types: minute_id, date, meeting_type, location, begin_time, end_time, participant
Training Framework: PyTorch + Hugging Face Transformers
Evaluation Metric: Micro F1-score, Precision and Recall (seqeval)

How It Works

The model assigns a label to each token in the input sequence, using the BIO scheme (Begin–Inside–Outside).
It can recognize and extract structured metadata from free-form text, even when expressed with stylistic variation.

python

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

# Load model and tokenizer
MODEL_NAME = "anonymous13542/BERTimbau-large-metadata-council-pt"
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForTokenClassification.from_pretrained(MODEL_NAME)
model.eval()

# Example text
text = "ATA Nº 23 REUNIÃO ORDINÁRIA PÚBLICA DA CÂMARA MUNICIPAL DE CAMPO MAIOR, REALIZADA EM 17 DE NOVEMBRO DE 2021.\nAos dezassete dias do mês de novembro do ano de dois mil e vinte e um, no Edifício dos Paços do Concelho, nesta Vila, realizou-se, pelas nove horas e trinta minutos, a reunião Ordinária Pública da Câmara Municipal, comparecendo os Excelentíssimos Senhores Luis Fernando Martins Rosinha, Paulo Ivo Sabino Martins de Almeida, Paulo Jorge Furtado Pinheiro, Maria da Encarnação Grifo Silveirinha e Fátima do Rosário Pingo Vitorino Pereira, respetivamente, Presidente e Vereadores efetivos deste Órgão Autárquico.\n-Verificada a presença dos respectivos membros, o Senhor Presidente declarou aberta a reunião:\n-Estava presente o Chefe **************************************, Dr. *********************************** e a Assistente Técnico **************************************.\n\n-ORDEM DO DIA:\n-INICIANDO A ORDEM DO DIA, ENTREGUE A TODO O EXECUTIVO E ELABORADA NOS TERMOS DO ARTIGO 53º DA LEI Nº 75/2013, DE 12 DE SETEMBRO, A CÂMARA TRATOU OS SEGUINTES ASSUNTOS:\n-ORDEM DO DIA:\n-FINANÇAS MUNICIPAIS:\n"

# Tokenize with offset mapping
inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    max_length=512,
    return_offsets_mapping=True
)
offsets = inputs.pop("offset_mapping")[0]

# Predict
with torch.no_grad():
    outputs = model(**inputs)

predictions = torch.argmax(outputs.logits, dim=2)[0]
labels = [model.config.id2label[p.item()] for p in predictions]

# Extract entities using character spans
entities = []
current = None

for (start, end), label in zip(offsets.tolist(), labels):
    if label == "O" or start == end:
        if current:
            entities.append(current)
            current = None
        continue

    if label.startswith("B-"):
        if current:
            entities.append(current)
        current = {"label": label[2:], "start": start, "end": end}
    elif label.startswith("I-") and current and label[2:] == current["label"]:
        current["end"] = end
    else:
        if current:
            entities.append(current)
        current = None

if current:
    entities.append(current)

# Print results
print("\nDetected Entities:")
for ent in entities:
    span = text[ent["start"]:ent["end"]]
    print(f"- {ent['label']}: {span}")

Evaluation Results

Municipal Meeting Minutes Test Set

Metric	Score
F1 score	0.96
Precision	0.95
Recall	0.97

Limitations

Domain Specificity: Best performance on administrative/governmental meeting minutes
Language: Optimized for Portuguese; English performance may vary
Context Window: Limited to 512 tokens

License

This project uses a custom dual-license based on AGPL v3.

See the full license terms here: LICENSE

inesctec/CitiLink-BERTimbau-large-metadata-pt-baseline

Baseline_BERTimbau-large-metadata-council-pt: Metadata Extraction for Municipal Meeting Minutes

Model Description