If you find this useful, please ⭐ star the repo and share with others!
Created: November 2025 | Format: LoRA Adapter (8-bit quantized base)
Model Description
CardioEmbed is a domain-specialized embedding model fine-tuned on comprehensive cardiology textbooks for clinical applications. Built on Qwen3-Embedding-8B using LoRA adapters, this model achieves state-of-the-art performance on biomedical retrieval tasks while maintaining efficiency through 8-bit quantization.
Why CardioEmbed?
Cardiovascular disease remains the leading cause of death globally, accounting for approximately 18 million deaths annually and representing nearly one-third of all mortality worldwide. In the United States alone, cardiovascular disease imposes an estimated annual economic burden exceeding $400 billion in direct medical costs and lost productivity.
As machine learning systems increasingly support clinical decision-making in cardiology—from risk stratification and diagnostic assistance to treatment optimization—the quality of semantic text representations becomes critical. However, existing biomedical embedding models trained primarily on PubMed research literature may not fully capture the procedural knowledge and specialized terminology found in clinical cardiology textbooks that practitioners actually use.
CardioEmbed bridges this research-practice gap by training on comprehensive cardiology textbooks, achieving near-perfect retrieval accuracy on cardiac-specific tasks while maintaining strong performance on general biomedical benchmarks.
Key Features
🏥 Medical Domain Expertise: Trained on 106,432 cardiology-specific sentence pairs from authoritative textbooks
🎯 Superior Performance: 26.4% improvement over base model on biomedical benchmarks
⚡ Efficient: LoRA adapters (117MB) + 8-bit quantization for production deployment
🔬 Research-Backed: Peer-reviewed methodology with comprehensive evaluation
Performance Highlights
Benchmark
CardioEmbed
Qwen3-8B Base
Improvement
BIOSSES
89.3%
82.1%
+7.2%
SciFact
72.4%
68.9%
+3.5%
NFCorpus
38.7%
34.2%
+4.5%
Avg MRR
66.8%
61.7%
+5.1%
MRR@10 on biomedical retrieval tasks. See paper for full results.
Performance Visualization
CardioEmbed achieves 99.60% Acc@1 on cardiac-specific retrieval, outperforming MedTE (current SOTA medical embedding) by +15.94 percentage points:
Model Comparison
Figure: Comparison of CardioEmbed against state-of-the-art medical and general-purpose embedding models on cardiology retrieval tasks.
Quick Start
Installation
pip install transformers peft torch
Basic Usage
python
1from transformers import AutoModel, AutoTokenizer
2from peft import PeftModel
3import torch
45# Load base model and CardioEmbed adapter6base_model = AutoModel.from_pretrained(7"Qwen/Qwen3-Embedding-8B",8 trust_remote_code=True,9 device_map="auto"10)11model = PeftModel.from_pretrained(base_model,"richardyoung/CardioEmbed")12tokenizer = AutoTokenizer.from_pretrained(13"Qwen/Qwen3-Embedding-8B",14 trust_remote_code=True15)1617# Generate embeddings for cardiology text18texts =[19"Acute myocardial infarction with ST-segment elevation",20"Patient presents with severe chest pain and dyspnea"21]2223defget_embeddings(texts):24 inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")25 inputs ={k: v.to(model.device)for k, v in inputs.items()}2627with torch.no_grad():28 outputs = model(**inputs)29# EOS token pooling (last token)30 embeddings = outputs.last_hidden_state[:,-1,:]3132return embeddings
3334embeddings = get_embeddings(texts)35print(f"Embedding shape: {embeddings.shape}")# [2, 4096]3637# Compute cosine similarity38similarity = torch.nn.functional.cosine_similarity(39 embeddings[0:1], embeddings[1:2]40)41print(f"Similarity: {similarity.item():.4f}")
Semantic Search Example
python
1# Clinical query and candidate documents2query ="What are the diagnostic criteria for heart failure?"3documents =[4"Heart failure diagnosis requires echocardiographic evidence of reduced ejection fraction",5"Hypertension management includes lifestyle modifications and pharmacotherapy",6"Atrial fibrillation treatment options include rate and rhythm control strategies"7]89# Embed query and documents10query_emb = get_embeddings([query])11doc_embs = get_embeddings(documents)1213# Rank by similarity14similarities = torch.nn.functional.cosine_similarity(15 query_emb.expand(len(documents),-1), doc_embs
16)17ranked_indices = similarities.argsort(descending=True)1819for idx in ranked_indices:20print(f"Rank {idx+1}: {documents[idx]} (score: {similarities[idx]:.4f})")
Training Details
Training Data
Source: Comprehensive cardiology textbooks (copyrighted, not publicly available)
Dataset Size: 106,432 semantically related sentence pairs
InfoNCE Contrastive Loss with temperature scaling (τ=0.05):
L = -log(exp(sim(zi, zj)/τ) / Σ exp(sim(zi, zk)/τ))
Where positive pairs are semantically related cardiology sentences, and in-batch negatives provide hard negative mining.
Evaluation
CardioEmbed was evaluated on the MTEB (Massive Text Embedding Benchmark) biomedical subset:
Biomedical Benchmarks
Task
Metric
CardioEmbed
Qwen3-8B
PubMedBERT
BioLinkBERT
BIOSSES
Spearman ρ
89.3%
82.1%
84.7%
86.2%
SciFact
NDCG@10
72.4%
68.9%
70.1%
71.3%
NFCorpus
NDCG@10
38.7%
34.2%
36.5%
37.8%
Retrieval Performance (MRR@10)
Cardiology-specific queries: 66.8% (+8.3% over base model)
General biomedical queries: 61.7% (+5.1% over base model)
Zero-shot transfer: Strong performance on unseen medical domains
See the full paper for comprehensive evaluation results.
Intended Use
Primary Applications
✅ Clinical Decision Support
Semantic search over medical literature
Patient case similarity matching
Clinical guideline retrieval
✅ Medical Information Retrieval
Biomedical question answering
Literature review automation
Evidence-based medicine workflows
✅ Healthcare NLP Pipelines
Document clustering and classification
Medical concept normalization
Clinical note analysis
Limitations
⚠️ Important Considerations
Domain Specificity: Optimized for cardiology; performance may vary on other medical specialties
Not a Diagnostic Tool: This model provides embeddings for information retrieval, not clinical diagnoses
Training Data: Trained on textbook knowledge; may not reflect latest clinical guidelines
Language: English only
Validation Required: All clinical applications require expert validation
Model Card Authors
Richard J. Young¹ and Alice M. Matthews²
¹ University of Nevada Las Vegas, Department of Neuroscience
² Concorde Career College, Department of Cardiovascular and Medical Diagnostic Sonography
Citation
If you use CardioEmbed in your research, please cite:
bibtex
1@article{young2025cardioembed,
2 title={CardioEmbed: Domain-Specialized Text Embeddings for Clinical Cardiology},
3 author={Young, Richard J. and Matthews, Alice M.},
4 journal={arXiv preprint arXiv:XXXX.XXXXX},
5 year={2025},
6 url={https://arxiv.org/abs/XXXX.XXXXX}
7}
License
This model is released under the Apache 2.0 License.
