is a model initialized with roberta-large and trained with Schizophrenia Reddit, a subset of Self-Reported Mental Health Diagnoses (SMHD) dataset which consists of Reddit posts by patients with schizophrenia only or schizophrenia with other mental disorders and matched control. We follow the standard pretraining protocols of RoBERTa with Huggingface’s Transformers library.

from transformers import AutoTokenizer,
AutoModel tokenizer = AutoTokenizer.from_pretrained("Amalq/schizophrenia-roberta-large")
model = AutoModel.from_pretrained("Amalq/schizophrenia-roberta-large")

Perplexity of this model is: 4.43