These results were produced in the local competition workspace and should be treated as project-side reporting rather than an official leaderboard submission.

Quick start

pip install torch transformers safetensors

python

from transformers import AutoTokenizer, AutoModelForMaskedLM

model_id = "YOUR_HF_USERNAME_OR_ORG/chinese-babylm-bert-mlm-first"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForMaskedLM.from_pretrained(model_id)
model.eval()

Use as an encoder for finetuning

python

from transformers import AutoModel, AutoTokenizer

model_id = "YOUR_HF_USERNAME_OR_ORG/chinese-babylm-bert-mlm-first"

tokenizer = AutoTokenizer.from_pretrained(model_id)
encoder = AutoModel.from_pretrained(model_id)
encoder.eval()

If you want to start a classification model from this checkpoint:

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "YOUR_HF_USERNAME_OR_ORG/chinese-babylm-bert-mlm-first"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id, num_labels=2)

The classification head created above is randomly initialized and must be finetuned on the target task.

Notes

The export excludes intermediate training checkpoints, optimizer states, and scheduler states.
The package is intended for distribution and reuse, not for exact training-state restoration.

NanceTide/chinese-babylm-bert-mlm-first

Chinese BabyLM First Model

Model summary

Files

Intended use

Evaluation snapshot

Quick start

Use as an encoder for finetuning

Notes

NanceTide/
chinese-babylm-bert-mlm-first