Views
No views yet
fawiki dump (WikiExtractor, spaCy blank("fa")
tokenization): 8,428,449 sentences, 190,781,621 tokens.--model cbow --mode floret --dim 300 --minn 5 --maxn 5 \
--hashCount 2 --bucket 200000 --neg 10 --epoch 5 --lr 0.05 \
--minCount 20 --batch 8192fa_dep_news_lg / fa_core_news_lg / fa_ent_news_lg
pipelines are trained against.python -m spacy init vectors fa fa.floret ./fa_pipeline --mode floretminCount 20. No OOV lookup.
1from gensim.models import KeyedVectors
2kv = KeyedVectors.load_word2vec_format("fa.vec")pipeline: []) with fa.floret embedded as a 200,000-row
x 300d table.
pip install https://huggingface.co/Phazel/fa-floret-wiki-vectors/resolve/main/fa_floret_wiki_200k-0.1.0-py3-none-any.whl1import spacy
2nlp = spacy.load("fa_floret_wiki_200k")
3print(nlp.vocab.vectors.shape) # (200000, 300)--paths.vectors at an unpacked copy
(scripts/unpack_vectors.py in
spacy-persian).| Package | Rows | Corpus | Used by |
|---|---|---|---|
fa_floret_400k | 50k | 400k Persian documents | md tier |
fa_floret_full_wiki | 50k | full Persian Wikipedia dump | — |
fa_floret_wiki_200k (this repo) | 200k | full Persian Wikipedia dump, 5 epochs | lg tier |
lg pipelines' meta.json.
The floret-torch training tooling is MIT and imposes no condition on the
output.