Stemm-compression experiment. Per-author variabiliy is interesting:
Zhitija_svjatykh:
0.86% 556 MiB => 4.81 MiB
world:
0.05% 1.98 MiB => 1.10 KiB
Vasilij_Bandakov:
0.62% 169 MiB => 1.05 MiB
Nikodim_Svjatogorets:
0.13% 1.64 GiB => 2.18 MiB
Manuil_Lemeshevskij:
4.78% 17.3 MiB => 844 KiB
Feodor_Studit:
0.06% 1.79 GiB => 1.17 MiB
Filaret_Moskovskij:
0.56% 549 MiB => 3.09 MiB
Avrelij_Avgustin:
0.73% 202 MiB => 1.47 MiB
Vasilij_Velikij:
0.28%… See the full description on the dataset page:
https://huggingface.co/datasets/foss22/azbu_corpus.
