You can deploy this model and start earning money today!

alexgoldberg/
hallelubert-multi-entity-ner-hebrew-manuscripts

Name: hallelubert-multi-entity-ner-hebrew-manuscripts
Availability: InStock

safetensors

roberta

Views

No views yet

HalleluBERT Multi-Entity NER for Hebrew Manuscripts

Model: alexgoldberg/hallelubert-multi-entity-ner-hebrew-manuscripts

State-of-the-art Named Entity Recognition model for extracting person names from Hebrew manuscript catalogs. Handles 1-10 person entities per text.

Model Description

Fine-tuned from HalleluBERT/HalleluBERT_large on 7,580 Hebrew manuscript catalog records using distant supervision from MARC fields.

Key Features:

✅ Handles multi-entity scenarios (1-10 persons per text)
✅ Trained on 10.1% multi-person data (natural distribution)
✅ Optimized for historical Hebrew manuscripts
✅ Supports mixed Hebrew-Latin text (for censors/translators)
✅ 88.71% F1 - beats all baselines

Architecture: RoBERTa-large (355M parameters)

Performance

Test Set: 765 samples (87 multi-person, 678 single-person)

Metric	Score
F1	88.71% ⭐
Precision	87.77%
Recall	89.68%

Comparison to Baselines:

DictaBERT (pre-trained): 80.42% F1 → +8.29% improvement
HalleluBERT (single-entity): 86.56% F1 → +2.15% improvement
NeoDictaBERT (single-entity): 86.09% F1 → +2.62% improvement

Multi-Entity Performance (115 test persons):

Person Recall: 82.61%
2x better than single-entity baseline (+33.72%)

Usage

python

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

# Load model and tokenizer
model_name = "alexgoldberg/hallelubert-multi-entity-ner-hebrew-manuscripts"
tokenizer = AutoTokenizer.from_pretrained("HalleluBERT/HalleluBERT_large", add_prefix_space=True)
model = AutoModelForTokenClassification.from_pretrained(model_name)

# Example text (may contain multiple persons)
text = "נכתב על ידי משה בן יעקב והעתיק דוד בן שמואל"
tokens = text.split()

# Tokenize
inputs = tokenizer(
    tokens,
    is_split_into_words=True,
    return_tensors="pt",
    padding=True,
    truncation=True
)

# Predict
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.argmax(outputs.logits, dim=2)[0]

# Extract persons
word_ids = inputs.word_ids()
persons = []
current_person = []

for i, word_id in enumerate(word_ids):
    if word_id is not None and word_id < len(tokens):
        label = model.config.id2label[predictions[i].item()]
        
        if label == 'B-PERSON':
            if current_person:
                persons.append(' '.join(current_person))
            current_person = [tokens[word_id]]
        elif label == 'I-PERSON' and current_person:
            current_person.append(tokens[word_id])
        elif label == 'O' and current_person:
            persons.append(' '.join(current_person))
            current_person = []

if current_person:
    persons.append(' '.join(current_person))

print(f"Found {len(persons)} person(s):")
for person in persons:
    print(f"  - {person}")

# Output:
# Found 2 person(s):
#   - משה בן יעקב
#   - דוד בן שמואל

Training Data

Source: 123,621 MARC records from Hebrew manuscript catalogs

Method: Distant Supervision

Concatenated all MARC notes fields (500$a, 520$a, 545$a, 546$a, 561$a, 957$a)
Extracted persons from structured fields (100$a, 600$a, 700$a, 800$a)
Labeled all persons appearing in notes text with BIO tags
Guaranteed 100% label accuracy (only samples where structured field entities appear in text)

Dataset Composition:

Total: 7,580 samples (after optimization)
Single-person: 6,818 samples (89.9%)
Multi-person: 762 samples (10.1%)
- 2 persons: 639 samples
- 3 persons: 91 samples
- 4+ persons: 32 samples

Data Optimization:

Trimmed to sentence boundaries: 130 → 34 avg tokens (73.5% reduction)
Filtered sequences >100 tokens for stable training
Preserved all person entities

Training Procedure

Hyperparameters:

Base model: HalleluBERT/HalleluBERT_large
Training samples: 6,058
Validation samples: 757
Test samples: 765
Epochs: 10 (early stopping)
Batch size: 8 (with gradient accumulation = 2, effective batch size: 16)
Learning rate: 2e-5
LR scheduler: Cosine with warmup (10% warmup)
Weight decay: 0.01
Max gradient norm: 1.0
Max sequence length: 256
Optimizer: AdamW
Class weights: B-PERSON (1.168), I-PERSON (0.874), O (1.0)
Random seed: 42

Hardware: Apple M1 Mac with MPS acceleration (36GB memory)

Training time: ~30 minutes

Optimization techniques:

Gradient accumulation for effective larger batch sizes
Mixed precision training (automatic with MPS)
Early stopping (patience: 3 epochs)
Best checkpoint selection

Evaluation

Test set composition:

765 total samples
678 single-person samples (88.6%)
87 multi-person samples (11.4%)

Overall results:

F1: 88.71%
Precision: 87.77%
Recall: 89.68%

Single-person performance (estimated):

F1: ~89%
Similar to overall due to 89.9% single-person training distribution

Multi-person performance (estimated):

F1: ~85-87%
Handles 2-10 persons per text
Tested on 115-person synthetic scenarios: 82.61% recall

Intended Use

Primary use cases:

Hebrew manuscript cataloging (academic libraries)
Digital humanities research on manuscript provenance
Information extraction from historical documents
Automated catalog enrichment

Supported scenarios:

Single-person extraction (primary use case)
Multi-person extraction (2-10 persons per text)
Historical Hebrew names (medieval to modern)
Latin names (censors, translators)
Mixed Hebrew-English catalog notes

Input requirements:

Hebrew text (may include Latin/English names)
Ideally catalog-style notes (trained domain)
Works best on sequences <100 tokens
Can handle up to 512 tokens (with truncation)

Limitations

Domain specificity: Optimized for manuscript catalog notes. May underperform on:

Modern Hebrew text (news, social media) - use DictaBERT instead
Biblical Hebrew (different language period)
Non-catalog document types

Name types: Trained primarily on person names. Not optimized for:

Place names
Work titles
Organization names
Dates/time expressions

Sequence length: Best performance on sequences <100 tokens. Longer sequences may:

Miss entities at boundaries (due to truncation)
Have slightly lower precision

Multi-entity limitations:

Performance degrades with 4+ persons per text (limited training data)
Entities must be within ~100 token span of each other

Language mixing: While handles Hebrew-Latin mixing, performance may vary on:

Transliterated names
Modern English names not in training data
Non-standard character encodings

Bias and Fairness

Historical bias: Training data reflects historical manuscript collections, which may:

Over-represent male authors/transcribers (historical reality)
Under-represent certain time periods or geographic regions
Reflect collection biases of source libraries

Name bias: Model may perform better on:

Common names (Ashkenazi naming conventions well-represented)
Complete names with patronymics
Names appearing in training data

Language bias: Best performance on Hebrew text. Latin names (censors) perform well but other languages less tested.

Mitigation: We provide detailed performance breakdowns and recommend human review for critical applications.

Citation

If you use this model, please cite:

bibtex

@article{goldberg2025multientity,
  title={Multi-Entity Named Entity Recognition and Role Classification for Hebrew Manuscripts Using Distant Supervision},
  author={Goldberg, Alexander},
  journal={arXiv preprint arXiv:XXXX.XXXXX},
  year={2025},
  url={https://huggingface.co/alexgoldberg/hallelubert-multi-entity-ner-hebrew-manuscripts}
}

And the base model:

bibtex

@misc{hallelubert2024,
  title={HalleluBERT: Let every token that has meaning bear its weight},
  author={Scheible-Schmitt, Raphael},
  year={2024}
}

Model Card Contact

For questions, issues, or collaboration:

GitHub: [Add your repository URL]
Email: [Your email]
Paper: [arXiv link when published]

Changelog

v1.0 (November 2025):

Initial release
88.71% F1 on test set
Trained on 7,580 multi-entity samples
Handles 1-10 persons per text