You can deploy this model and start earning money today!

InfocubeSrl/
LexCube

dbmdz/bert-base-italian-uncased

finetune

apache-2.0

endpoints_compatible

Views

No views yet

Model Card for Model ID

This model is a BERT-based Masked Language Model fine-tuned on Italian legal texts (municipal delibera domain*). It is designed to predict masked tokens in legal documents and capture domain-specific semantic and syntactic structures.

*A delibera is a formal decision or resolution made by a local government body, like a city council or municipal committee, that has official and legal effect.

Model Description

This model is fine-tuned from dbmdz/bert-base-italian-uncased using Masked Language Modeling (MLM) with Whole Word Masking (WWM).
WWM ensures that all subword tokens of a selected word are masked together, encouraging the model to learn deeper contextual representations, especially for complex legal terminology.

Developed by: Mohammad Mahdi Heydari Asl / infocube
Model type: Transformer, BERT-based Masked Language Model
Language(s): Italian
License: Apache-2.0
Finetuned from model: dbmdz/bert-base-italian-uncased

Uses

Direct Use

The model can be used for:

Predicting masked tokens in Italian legal texts ([MASK] prediction)
Embedding legal text for downstream NLP tasks
Transfer learning for other Italian legal NLP applications

Bias, Risks, and Limitations

Not suitable for general-purpose Italian NLP outside legal text.

Recommendations

Users should verify outputs and avoid relying on predictions for legal decision-making without expert supervision.

How to Get Started with the Model

python

from transformers import AutoTokenizer, AutoModelForMaskedLM
import torch

model_name = "InfocubeSrl/LexCube"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForMaskedLM.from_pretrained(model_name)

# Examples with [MASK]
examples = [
    "[MASK] il Decreto Legislativo 18 agosto 2000, n. 267 (Testo Unico delle leggi sull'ordinamento degli Enti Locali)",
    "ACQUISITI, ai sensi dell'art. [MASK] del D.Lgs. 267/2000, i pareri favorevoli di regolarità tecnica e di regolarità contabile",
    "Visto gli art. [MASK] e 42 del D.Lgs n.267/2000, Testo unico degli enti locali.",
    "DI DICHIARARE la presente deliberazione immediatamente [MASK] ai sensi dell'art. 134, comma 4, del D.Lgs. n. 267/2000."
]

for text in examples:
    inputs = tokenizer(text, return_tensors="pt")
    outputs = model(**inputs)

    # Find mask token position
    mask_index = (inputs["input_ids"][0] == tokenizer.mask_token_id).nonzero(as_tuple=True)[0]
    
    # Get top prediction
    predicted_id = outputs.logits[0, mask_index].argmax(dim=-1)
    predicted_token = tokenizer.decode(predicted_id)

    print(f"Input: {text}")
    print(f"Prediction: {predicted_token}\n")

Training Data

Source: Provided by Infocube,
Size: 15,646 documents
Language: Italian
Domain: Legal and administrative texts (municipal delibera domain)
- Formal and technical legal language
- Frequent references to laws, decrees, and legislative articles
- Structured format with numbered provisions and cross-citations
- Avg. length: ~909 words (≈2,193 tokens per document); some documents exceed 11k tokens
Confidentiality: Raw dataset cannot be shared due to contractual agreements, but it has been statistically and linguistically analyzed for research