You can deploy this model and start earning money today!

zymonody/
chinese-babylm-v4

Name: chinese-babylm-v4
Rating: 2.5 (1036 reviews)

Views

No views yet

Chinese BabyLM 2026 submission (v4)

A Chinese masked language model for the NLPCC 2026 Chinese BabyLM Challenge, pretrained within the ≤102M Jieba-word data budget.

Architecture: DeBERTa-v2 base (12 layers, hidden 768, 12 heads), ~102M params
Tokenizer: character-level
Objective: masked language modeling

Methodology details will be released in the technical report after the evaluation period.

Usage

python

from transformers import AutoTokenizer, AutoModelForMaskedLM
tok = AutoTokenizer.from_pretrained("zymonody/chinese-babylm-v4")
model = AutoModelForMaskedLM.from_pretrained("zymonody/chinese-babylm-v4")