aisingapore/

SEA-PILE-v1

SEA-LION-Pile is the pretraining data set for SEA-LION, a collection of Large Language Models (LLMs) which has been pretrained and instruct-tuned for the Southeast Asia (SEA) region. This repository contains the cleaned mC4 portion of the SEA-LION-Pile. For the remainder of the SEA-LION-Pile dataset, they may be downloaded from the links provided below.

SEA-LION was trained on 980B tokens of the following data:

Data… See the full description on the dataset page: https://huggingface.co/datasets/aisingapore/SEA-PILE-v1.