You can deploy this model and start earning money today!

krasserm/
perceiver-io-txt-clf-imdb

Name: perceiver-io-txt-clf-imdb
Availability: InStock

transformers

pytorch

perceiver-io-text-classifier

text-classification

imdb

apache-2.0

Views

No views yet

Perceiver IO sentiment analysis model (IMDb)

This model is a Perceiver IO sentiment analysis model fine-tuned on the IMDb dataset. It is a training example of the perceiver-io library.

Model description

The configuration of the encoder is equal to the encoder configuration of krasserm/perceiver-io-mlm-imdb, the generic decoder is configured with a classification output adapter (see building blocks).

Intended use and limitations

The model can be used for sentiment analysis of movie reviews.

Model training

Training details are described here. Encoder weights are initialized with the encoder weights of krasserm/perceiver-io-mlm-imdb decoder weights are randomly initialized. In a first step, only decoder weights are trained, with encoder weights frozen. In a second step, all weights are fine-tuned. Training was done with PyTorch Lightning and the resulting checkpoint was converted to this 🤗 model with a library-specific conversion utility.

Usage examples

To use this model you first need to install the perceiver-io library with extension text.

pip install perceiver-io[text]

Then the model can be used with PyTorch. Either use the model and tokenizer directly

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from perceiver.model.text import classifier  # auto-class registration

repo_id = "krasserm/perceiver-io-txt-clf-imdb"

model = AutoModelForSequenceClassification.from_pretrained(repo_id)
tokenizer = AutoTokenizer.from_pretrained(repo_id)

mini_reviews = [
    "I've seen this movie yesterday and it was really boring.",
    "I can recommend this movie to all fantasy movie lovers.",
]

encoding = tokenizer(mini_reviews, padding=True, return_tensors="pt")
logits = model(**encoding).logits

predictions = logits.argmax(dim=-1)

for r, p in zip(mini_reviews, predictions.numpy()):
    print(f"{r} ({model.config.id2label[p]})")

I've seen this movie yesterday and it was really boring. (NEGATIVE)
I can recommend this movie to all fantasy movie lovers. (POSITIVE)

or use a sentiment-analysis pipeline:

python

from transformers import pipeline
from perceiver.model.text import classifier  # auto-class registration

repo_id = "krasserm/perceiver-io-txt-clf-imdb"

mini_reviews = [
    "I've seen this movie yesterday and it was really boring.",
    "I can recommend this movie to all fantasy movie lovers.",
]

sentiment_pipeline = pipeline("sentiment-analysis", model=repo_id)
predictions = sentiment_pipeline(mini_reviews)

for r, p in zip(mini_reviews, predictions):
    print(f"{r} ({p['label']})")

I've seen this movie yesterday and it was really boring. (NEGATIVE)
I can recommend this movie to all fantasy movie lovers. (POSITIVE)

Checkpoint conversion

The krasserm/perceiver-io-mlm-imdb model has been created from a training checkpoint with:

python

from perceiver.model.text.classifier import convert_imdb_classifier_checkpoint

convert_imdb_classifier_checkpoint(
    save_dir="krasserm/perceiver-io-txt-clf-imdb",
    ckpt_url="https://martin-krasser.com/perceiver/logs-0.8.0/txt_clf/version_1/checkpoints/epoch=006-val_loss=0.156.ckpt",
    tokenizer_name="krasserm/perceiver-io-mlm",
    push_to_hub=True,
)