DiffutronLM-Pretraining-Corpus is the comprehensive, filtered Turkish text dataset used during the Continual Pre-training (CPT) phase of the Diffutron language models.
The primary goal of this dataset was to align the cross-lingual representations of a multilingual base encoder (jhu-clsp/mmBERT-base) with the agglutinative complexity and morphological nuances of the Turkish language, without inducing catastrophic forgetting.
📊 Dataset… See the full description on the dataset page: https://huggingface.co/datasets/diffutron/DiffutronLM-Pretraining-Corpus.
