GNTweets
This folder contains the dataset used to train the model guaran-ia/gntweets-lm
Splits and sizes
train: 936 records (1,916,928 tokens)
validation: 117 records (239,616 tokens)
test: 117 records (239,616 tokens)
Format
train_dataset/
val_dataset/
test_dataset/
Loading
from datasets import load_dataset
ds = load_dataset('guaran-ia/gntweets')
train = ds['train']
