This is a fine-tuned version of albert-base-v2 on paraphrase detection tasks such as GLUE-QQP (Quora Question Pairs) and MRPC (Microsoft Research Paraphrase Corpus).
It can be used to determine whether two sentences are paraphrases (semantically similar) or not.

Developed by: Peeyush
Model type: Sentence-pair classification (binary: paraphrase vs not paraphrase)
Language(s): English
License: Apache-2.0
Finetuned from model: albert-base-v2

Model Sources [optional]

Repository: your-username/albert-paraphrase-similarity
Paper (base model): ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Uses

Direct Use

Paraphrase detection: Check if two sentences mean the same thing.
Semantic textual similarity: Determine closeness of meaning between two texts.

Downstream Use

Duplicate question detection (e.g., Q&A forums like Quora or StackOverflow).
Information retrieval (ranking by semantic similarity).
Chatbots / Virtual assistants (detecting intent rephrasing).

Out-of-Scope Use

Not a generative model → cannot rewrite or generate paraphrases.
Not trained on multilingual data → limited to English.

Bias, Risks, and Limitations

The model inherits biases from QQP/MRPC (e.g., common question styles, certain domains).
May not generalize to informal text, code-mixed text, or specialized domains (e.g., medical, legal).
Can misclassify edge cases where semantic similarity is subtle.

Recommendations

Always evaluate on your target domain before deployment.
For production, consider threshold-tuning (instead of raw classification).

How to Get Started with the Model

Example usage:

python

model  = AutoModelForSequenceClassification.from_pretrained('peeyush01/albert-paraphrase-detector')
tokenizer = AutoTokenizer.from_pretrained('peeyush01/albert-paraphrase-detector-tokenizer')

def predict_paraphrase(sentence1, sentence2):
    inputs = tokenizer(sentence1, sentence2, return_tensors="pt", padding=True, truncation=True)
    with torch.no_grad():
        outputs = model(**inputs)
        logits = outputs.logits
        probs = torch.softmax(logits, dim=1)
        paraphrase_prob = probs[0][1].item()
    return {"Paraphrase": paraphrase_prob, "Not Paraphrase": 1 - paraphrase_prob}

python

import torch

pairs = [
    ("The movie was fantastic!", "The film was amazing!"),
    ("He is playing cricket.", "She is reading a book."),
]

for s1, s2 in pairs:
    result = predict_paraphrase(s1, s2)
    print(f"Sentence 1: {s1}")
    print(f"Sentence 2: {s2}")
    print(f"Result: {result}\n")

Training Details

Training Data

Dataset: GLUE MRPC
Description: The Microsoft Research Paraphrase Corpus (MRPC) contains pairs of sentences automatically extracted from online news sources, with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.
Size: ~3,700 training pairs, 408 validation pairs, 1,725 test pairs.
Labels:
- 1 → Paraphrase (semantically equivalent)
- 0 → Not paraphrase

Training Procedure

Preprocessing

Both sentences were tokenized using AlbertTokenizer with truncation and padding (max_length).
Columns sentence1, sentence2, and idx were dropped.
The label column was renamed from label → labels.
Dataset was set in PyTorch format.

Training Hyperparameters

Base model: albert-base-v2
Epochs: 3
Batch size: 16 (train and eval)
Optimizer: AdamW (via Hugging Face Trainer)
Warmup steps: 600
Weight decay: 0.01
Evaluation strategy: Per epoch
Precision regime: FP32

Speeds, Sizes, Times

Training performed with Hugging Face Trainer.
Training time: ~20–30 mins on a single GPU (Tesla T4); longer on CPU.
Final checkpoint size: ~47 MB.

Evaluation

Testing Data, Factors & Metrics

Testing Data

Evaluation performed on the GLUE MRPC validation set (~408 examples).

Factors

Sentence pairs vary in length, syntactic complexity, and semantic overlap.
Evaluation primarily captures semantic similarity in short news-style English text.

Metrics

Accuracy: percentage of correctly classified sentence pairs.
F1 Score: harmonic mean of precision and recall, important due to class imbalance.

Results

(Expected range for ALBERT-base on MRPC — please replace with your actual run metrics if available)

Accuracy: ~86–88%
F1 Score: ~89–91%

Summary

The fine-tuned ALBERT model achieves strong performance on the MRPC benchmark, demonstrating effectiveness at capturing semantic similarity and paraphrase relationships between sentence pairs.