A simple implementation of FastText from
https://arxiv.org/pdf/1607.04606.
Made for beginner to reference and learn from. This repository contains tokenizer.json, model.safetensors, and train.ipynb.
The training workflow use modern libraries like HuggingFace's datasets and tokenizers. The rest is just PyTorch.
There are some algorithm which we decided to not implement which is the hashing function from the original paper.
The resulting training workflow is not as fast as the paper suggest, maybe it is because of the use of Python instead
of
Hogwild! instead.
