Welcome to Shiraz, the repository for Lifeweb's language model. First versions of our models are all trained on our own dataset called Divan with more than 164 million documents and more than 10B tokens which is normalized and deduplicated meticulously to ensure its enrichment and comprehensiveness. A better dataset leads to a better model!

You can easily access the models using the sample code provided below.

The Shiraz is evaluated on three downstream NLP tasks comprising NER, Sentiment Analysis, and Emotion Detection. Shiraz is considerably faster, and its accuracy remains highly competitive without compromising much on speed. According to MobileBERT paper, this model is 4.3× smaller and 5.5× faster than BERT-base.

Obvious from the table below, you can find the colab codes for each task to use as a tutorial besides the macro F1 score.

If you tested our models on a public dataset, and you wanted to add your results to the table above, open a pull request or contact us. Also make sure to have your code available online so that we can add a reference.

You are welcome to use our LM models in your work or research, if so, we kindly ask you to cite it using the following entry:

@misc{Shiraz,
    author = {Mehrdad Azizi, Reza Salehi Chegeni, Parisa Mousavi, Iman Hashemi},
    title = {[Optimizing Pre-trained BERT-based Models for Persian Language Processing]},
    year = {2024},
    publisher = {LifeWeb}
}

First version of Shiraz model trained on DIVAN.