nusnlp/

JGP-SlimPajama

The data repository for the SlimPajama data used in the following paper:

Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models Muhammad Reza Qorib, Junyi Li, and Hwee Tou Ng The 63rd Annual Meeting of the Association for Computational Linguistics (to appear)

Paper: arXiv Codebase: https://github.com/nusnlp/Just-Go-Parallel/

To download and extract the files, please run the following commands:… See the full description on the dataset page: https://huggingface.co/datasets/nusnlp/JGP-SlimPajama.