You can deploy this model and start earning money today!

mia-project-2025/
bert-base-uncased-adapter-quora-question-pairs

Name: bert-base-uncased-adapter-quora-question-pairs
Availability: InStock

safetensors

apache-2.0

Views

No views yet

Prefix-Tuned BERT for Quora Duplicate Question Detection

This repository contains a Prefix-Tuned BERT model fine-tuned on the Quora Duplicate Questions Dataset.
The task is binary classification: given two questions, predict whether they are duplicates.

Model Details

Base Model: tomaarsen/bert-base-nq-prompts
Fine-tuning Method: PEFT with Prefix-Tuning
Task: Sequence Classification (Duplicate vs. Not Duplicate)
Frameworks: Hugging Face Transformers, PEFT, PyTorch

Training Configuration

Epochs: 15
Batch Size: 32 (train) / 16 (eval)
Learning Rate: 5e-5 (AdamW fused optimizer)
Scheduler: Cosine
Prefix-Tuning Parameters:
- Virtual tokens: 20
- Projection: Enabled
- Layers: All encoder layers

Dataset

Name: Quora Duplicate Questions Dataset
Train/Test Split: 80/20
Task: Identify if two questions are semantically duplicate.

Results

Model	Dataset	Epochs	Eval Loss	Accuracy	Precision	Recall	F1	Train Loss	Train Runtime	Eval Runtime
Prefix-Tuned BERT (tomaarsen/bert-base-nq-prompts)	Quora	15	0.2891	0.8763	0.8070	0.8720	0.8382	0.3222	1664.56s	19.59s

Usage

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_name = "mia-project-2025/bert-base-uncased-adapter-quora-question-pairs"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Example input
questions = ["How do I learn Python?", "What is the best way to learn Python?"]
inputs = tokenizer(questions[0], questions[1], return_tensors="pt", padding=True, truncation=True)

# Prediction
outputs = model(**inputs)
pred = outputs.logits.argmax(-1).item()
print("Duplicate" if pred == 1 else "Not Duplicate")