This datasets hosts a sentence-tokenized version of the BIOfid dataset.
The following script can be used to reproduce the creation of the dataset:
import json
with open("./train.jsonl", "wt") as f_out:
for sentence in corpus.train:
current_example = {
"text": sentence.to_tokenized_string()
}… See the full description on the dataset page:
https://huggingface.co/datasets/german-tokenizer-benchmark/biofid.