It was made with the purpose of being able to classify scholarly work with a fixed set of well-defined topics. This is NOT the full model being used to tag OpenAlex works with a topic. For that, check out the following github repo:
OpenAlex Topic Classification
That repository will also contain information about text preprocessing, modeling, testing, and deployment.
Model description
The model was trained using the following input data format (so it is recommended the data be in this format as well):
Using both title and abstract:
"<TITLE> {insert-processed-title-here}\n<ABSTRACT> {insert-processed-abstract-here}"
Using only title:
"<TITLE> {insert-processed-title-here}"
Using only abstract:
"<TITLE> NONE\n<ABSTRACT> {insert-processed-abstract-here}"
The quickest way to use this model in Python is with the following code (assuming you have the transformers library installed):
from transformers import pipeline
title = "{insert-processed-title-here}"
abstract = "{insert-processed-abstract-here}"
classifier = \
pipeline(model="OpenAlex/bert-base-multilingual-cased-finetuned-openalex-topic-classification-title-abstract", top_k=10, "truncation":True,"max_length":512)
classifier(f"""<TITLE> {title}\n<ABSTRACT> {abstract}""")
This will return the top 10 outputs from the model. There will be 2 pieces of information here:
Full Topic Label: Made up of both the OpenAlex topic ID and the topic label (ex: "1048: Ecology and Evolution of Viruses in Ecosystems")
Model Score: Model's confidence in the topic (ex: "0.364")
Intended uses & limitations
The model is intended to be used as part of a larger model that also incorporates journal information and citation features. However, this model is good if you want to use it for quickly generating a topic based only on a title/abstract.
Since this model was fine-tuned on a BERT model, all of the biases seen in that model will most likely show up in this model as well.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
