chinese-babylm-cog-a197-strict-best is a custom BERT-based wrapper model for the NLPCC 2026 Chinese BabyLM shared task, optimized for the Cognitive Modeling (Cog) Track. It combines internal strict BERT branches, official-corpus static features, and MLM uncertainty features for brain-aligned fMRI evaluation.

Competition Compliance

Requirement	Status
From scratch	Source weights from internal random-initialized strict lineage only
No pretrained checkpoint	No official baseline or external pretrained checkpoint
No distillation	No teacher-model distillation
Data option	Official corpus (`chinese-babylm-org/babylm-zho-100M`)
Evaluation leakage	No CogBench stimulus text, labels, predictions, or item-level feedback used in training
No external resources	No pypinyin, IDS/CJKVI, radical tables, glyph/font resources, or inference-time rules
Frozen submission	Single exported wrapper checkpoint

Architecture

Field	Value
Model type	`CogAllLayerUncertaintyGate` wrapper
Architecture	`CogAllLayerUncertaintyGateModel`
Backend	`mlm`
Hidden size	1409
Max positions	256
Vocabulary	16000 (WordPiece, trained on official corpus)

Components

Word branch: A180 strict word BERT, layers [2,3,4] concat, RI scale 3.05 + log-frequency features.

Sentence branch: A130 + gamma -5.1 × (A158_s0250 − A122_s1500), layers [6,7,8] weighted [0, 0.98, 0.02] + raw384 window8 + log-frequency features.

Uncertainty: A129 step_01000 MLM features (logprob, surprisal, entropy, margin, probability), scale 1.25.

Hidden Evaluation

Evaluated with the final Chinese BabyLM pipeline (chinese-babylm/chinese-babylm-pipeline-final):

Task	Score
word_fmri	0.5655
fmri	0.1138

Usage

python

from transformers import AutoModel, AutoTokenizer

repo_id = "LikC1606/chinese-babylm-cog-a197-strict-best"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModel.from_pretrained(repo_id, trust_remote_code=True, output_hidden_states=True)

inputs = tokenizer("春天的花开了。", return_tensors="pt")
outputs = model(**inputs, output_hidden_states=True)

Limitations

Feature extraction model only, not for text generation.
Requires trust_remote_code=True.
Optimized for Cog track; not intended as a general NLU model.