You can deploy this model and start earning money today!

Bam3752/
PubMedBERT-BioNLI-LoRA

Name: PubMedBERT-BioNLI-LoRA
Availability: InStock

Views

No views yet

PubMedBERT BioNLI LoRA

License: MIT

PubMedBERT BioNLI LoRA is a biomedical Natural Language Inference (NLI) model fine-tuned with LoRA adapters.
It classifies entailment, contradiction, and neutrality between biomedical text pairs, optimized for chain-of-thought reasoning validation.

📊 Training Details

Base model: pritamdeka/PubMedBERT-MNLI-MedNLI
Fine-tuning datasets: BioASQ + MedNLI
Objective: 3-class NLI (entailment / neutral / contradiction)
Method: LoRA parameter-efficient fine-tuning
Hardware: Apple MPS (Metal backend)
Hyperparameters:
- Epochs: 4
- Learning rate: 1e-5
- Batch size: 8
- Max length: 256
- Gradient accumulation: 2
- Warmup ratio: 0.1
- Label smoothing: 0.05

📈 Results

Metric	Value
Accuracy	90.39%
Macro F1	0.9036
Eval Loss	0.2673

👉 Calibrated with isotonic regression (calibration/isotonic.pkl) for reliable probabilities.

🚀 Usage

Transformers

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("Bam3752/PubMedBERT-BioNLI-LoRA")
tokenizer = AutoTokenizer.from_pretrained("Bam3752/PubMedBERT-BioNLI-LoRA")

premise = "Aspirin reduces the risk of myocardial infarction."
hypothesis = "Aspirin prevents heart attacks."

inputs = tokenizer(premise, hypothesis, return_tensors="pt")
outputs = model(**inputs)

probs = outputs.logits.softmax(-1).detach().cpu().numpy()
print(probs)  # [neutral, contradiction, entailment]