Views
No views yet
1from sentence_transformers import SentenceTransformer
2
3# Load the model
4model = SentenceTransformer("lokeshch19/ModernPubMedBERT")
5
6# Encode medical texts
7texts = [
8 "Rheumatoid arthritis is an autoimmune disorder attacking joint linings.",
9 "Inflammatory cytokines in RA lead to progressive cartilage and bone destruction."
10]
11embeddings = model.encode(texts)NeuML/bioclinical-modernbert-base-embeddings, our model demonstrates superior understanding of medical concepts and enhanced discrimination of non-medical content.| Metric | Our Model | NeuML Model | Improvement |
|---|---|---|---|
| Accuracy@1 | 91.28% | 85.86% | +6.3% |
| Accuracy@3 | 98.46% | 95.66% | +2.9% |
| Accuracy@5 | 99.24% | 97.14% | +2.2% |
| Accuracy@10 | 99.64% | 98.29% | +1.4% |
| NDCG@5 | 95.96% | 92.37% | +3.9% |
| NDCG@10 | 96.10% | 92.75% | +3.6% |
| MRR@10 | 94.89% | 90.90% | +4.4% |
| MAP@100 | 94.91% | 90.96% | +4.3% |
InformationRetrievalEvaluator from sentence-transformers on the gamino/wiki_medical_terms dataset.NeuML/bioclinical-modernbert-base-embeddings, this model demonstrates superior understanding of medical concepts and enhanced discrimination of non-medical content.1text1 = "Hypertension increases the risk of stroke and heart attack."
2text2 = "High blood pressure damages arterial walls over time, leading to cardiovascular events."
3
4# Cosine Similarity Results:
5# Our Model: 0.5941 (59.4%)
6# NeuML Model: 0.5267 (52.7%)
7# Improvement: +12.7%1texts = ["diabetes type 2", "asyncio.run()"]
2
3# Cosine Similarity Results:
4# Our Model: 0.0804 (8.0%) - Correctly identifies low similarity
5# NeuML Model: 0.1926 (19.3%) - Higher false similarity
6# Better Discrimination: 58% lower false positive rate