Is this your dataset? Claim it with the Hugging Face account that owns it.

jhu-clsp/
mmBERT-decay-data

Name: mmBERT-decay-data
Creator: jhu-clsp
License: https://spdx.org/licenses/MIT.html

Views

No views yet

MMBERT Decay Phase Data

Phase 3 of 3: Annealed language learning decay phase (100B tokens) with massive multilingual expansion to 1833 languages.

📊 Data Composition

NOTE: there are multiple decay data mixtures: this mixture described below is the Decay-Cont mixture. However, the data in this repository is the Decay-Eng. If you are interested in the others, please let me know so I can prioritize it.

Data Source Tokens (B) Percentage Description

FineWeb2… See the full description on the dataset page: https://huggingface.co/datasets/jhu-clsp/mmBERT-decay-data.