from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name="agentlans/deberta-v3-xsmall-readability"

# Put model on GPU or else CPU
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

def readability(text):
    """Processes the text using the model and returns its logits.
    In this case, it's reading grade level in years of education
    (the higher the number, the harder it is to read the text)."""
    inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True).to(device)
    with torch.no_grad():
        logits = model(**inputs).logits.squeeze().cpu()
    return logits.tolist()

# Example usage
text = ["One day, Tim's teddy bear was sad. Tim did not know why his teddy bear was sad.",
        "A few years back, I decided it was time for me to take a break from my mundane routine and embark on an adventure.",
        "We also experimentally verify that simply scaling the pulse energy by 3/2 between linearly and circularly polarized pumping closely reproduces the soliton and dispersive wave dynamics."]
result = readability(text)
[round(x, 1) for x in result] # Estimated reading grades [2.9, 9.8, 21.9]

Performance metrics and training details

Performance Metrics

On the evaluation set:

Loss: 1.0767
Mean Squared Error (MSE): 1.0767

Training Procedure

Hyperparameters

Learning Rate: 5e-05
Train Batch Size: 8
Eval Batch Size: 8
Seed: 42
Optimizer: Adam (betas=(0.9, 0.999), epsilon=1e-08)
Learning Rate Scheduler: Linear
Number of Epochs: 3.0

Framework Versions

Transformers: 4.44.2
PyTorch: 2.2.2+cu121
Datasets: 2.18.0
Tokenizers: 0.19.1

Limitations

English only
Performance may vary for very long or very short texts
This model is for general texts so it's not optimized for specific uses like children's books or medical texts
Doesn't assess whether the texts make sense for the reader
There's a lot of variability in the readability metrics in the literature

Ethical Considerations

The model should not be the sole determinant for content suitability decisions
- The writer or publisher should also consider the content, context, and reader expectations
Potential social or societal biases due to the training data sources