# themes = ['баги', 'открытие', 'баланс', 'рейтинг', 'ревизия', 'другое']

from transformers import AutoTokenizer, AutoModel
import torch
model_name = 'wyluilipe/wb-themes-classification'

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = BertForSequenceClassification.from_pretrained(model_name, num_labels=i+1)

text = "программа не работает"
encoded_input = tokenizer(text, return_tensors='pt')

with torch.no_grad():
   output = model(**encoded_input)
   probabilities = torch.nn.functional.softmax(output.logits, dim=-1)
   predicted_class = torch.argmax(probabilities).item()

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	60	0.7383	0.8404
No log	2.0	120	0.8743	0.7840
No log	3.0	180	0.7312	0.8169
No log	4.0	240	0.6733	0.8404
No log	5.0	300	0.7612	0.7981
No log	6.0	360	0.7671	0.8122
No log	7.0	420	0.7306	0.8263
No log	8.0	480	0.7523	0.8263
0.1118	9.0	540	0.7645	0.8263
0.1118	10.0	600	0.7461	0.8310

Framework versions

Transformers 4.37.1
Pytorch 2.1.2
Datasets 2.16.1
Tokenizers 0.15.1