knightnemo/

LMCompressionDataset

This is a dataset for testing the compression ratio of Large language models. We utilize three datasets for this purpose:

Math: HuggingFaceH4/MATH-500 Code: Qwen/CodeElo General Purpose: HuggingFaceTB/smollm-corpus

For the third dataset, because the dataset is relatively large, we subsample 500 samples from the dataset. For all datasets, we truncate the data til 2048 tokens.