1import spacy
2nlp = spacy.load("fa_core_news_md")3doc = nlp("شرکت ایران خودرو اعلام کرد که تولید خود را افزایش میدهد.")4print([(t.text, t.pos_, t.lemma_, t.dep_)for t in doc])5print([(e.text, e.label_)for e in doc.ents])
Accuracy
Scored with spacy benchmark accuracy on the held-out PerDT test split.
Metric
Score
Tokenization accuracy
99.96
XPOS tag accuracy
96.25
UPOS tag accuracy
96.64
Morphological features
96.64
Lemma accuracy
97.96
Unlabelled attachment (UAS)
90.52
Labelled attachment (LAS)
86.34
Sentence segmentation F
99.28
NER precision
76.56
NER recall
72.95
NER F-score
74.71
Throughput
Median of repeated nlp.pipe passes over the 146-document PerDT test
split (23,825 tokens), timing the pipe only. Warmup pass discarded.
Trained on UD_Persian-PerDT, licensed CC BY-SA 4.0; this pipeline is therefore distributed under CC BY-SA 4.0 with attribution to the treebank authors. The ner component is trained on the NER layer shipped in UD_Persian-PerDT's not-to-release/ directory, so it shares the treebank's genre, tokenization and licence. Those labels are SILVER: the treebank README states they were produced by the BERT-based Beheshti-NER tagger (Taher et al., 2020) with manual corrections to extend recall. They were transferred onto this pipeline's tokenization by difflib alignment at a 99.86% transfer rate (scripts/transfer_perdt_ner.py); spans that could not be aligned exactly were dropped rather than guessed. Labels PER, LOC, ORG and DAT have 1,300 or more training examples each; MON (205), TIM (135) and PCT (121) are thin and their scores in performance.ents_per_type should be read before relying on them. Multiword tokens (pronominal clitics, enclitic copulas) were merged with spacy convert --merge-subtokens, so a small number of XPOS tags are composite (e.g. N_IANM_PR_JOPER) and ~1.5% of lemmas contain a space. doc.noun_chunks under-fires on this pipeline: spacy/lang/fa/syntax_iterators.py matches ClearNLP labels that do not exist in Universal Dependencies, see docs/upstream/fa-noun-chunks.md. This is the md tier: identical architecture to the sm pipeline plus static floret vectors (50,000 rows x 300 dimensions, minn=maxn=5, hash_count=2) trained on 400,000 Persian documents. floret hashes subwords into a fixed table, so there are no out-of-vocabulary tokens and token.has_vector is always True. That matters for Persian, where inconsistent ZWNJ (U+200C) usage splits one word across several surface forms (mi-ravad written joined, with ZWNJ, or with a space) that a classic word-vector table would miss.
