scaling-mmbert-10k (Stage A, warmed base)
Research checkpoint from the mmBERT head-init data-scaling curve (private).
mmBERT-base with fresh GLiNER2 heads (from_encoder) warmed on ~10,000
structure/argument-dense event records (multilingual, assembled from public event
corpora; RAMS/WikiEvents held out to avoid leakage), 5 epochs.
This is the N=10k point's warmed base — the reusable head-init artifact.
It is not fine-tuned on any single downstream task; fine-tune it (e.g. on RAMS)
to obtain a usable event model. The paired fine-tune is whr778/scaling-mmbert-10k-rams.
Purpose
Measures how much structure/argument data is needed to warm mmBERT's fresh
extraction heads before downstream fine-tuning (head-initialization thesis). See
the experiment spec SCALING_CURVE_EXPERIMENT.md.
- Encoder:
jhu-clsp/mmBERT-base (native long-context, 2048 training window)
- Stage-A corpus size: ~10,000 records
- Loss:
bce_posweight (pos_weight 4.0); differential-style LR (encoder 2e-5 / heads 5e-4)
Caveats
Research artifact, private, not benchmarked as a general model. Label quality is
bounded by the source event corpora. One point on a scaling curve, not a release.
