Is this your dataset? Claim it with the Hugging Face account that owns it.

orionweller/
mmBERT-pretrain-p2-fineweb2-remaining

Name: mmBERT-pretrain-p2-fineweb2-remaining
Creator: orionweller
License: https://spdx.org/licenses/MIT.html

Views

No views yet

mmBERT Pre-training Data P2

Phase 1 of 3: Diverse multilingual pre-training data mixture (trained for 2.3T tokens) used to train the mmBERT model suite.

NOTE: this is only P2 of the pre-training data due to HF limits, you need to download and combine all three into one folderThis dataset contains the pre-training phase data used to train all mmBERT encoder models. The data is provided in MDS format ready for use with Composer and the ModernBERT training repository.… See the full description on the dataset page: https://huggingface.co/datasets/orionweller/mmBERT-pretrain-p2-fineweb2-remaining.