StethoLM is the first audio–language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis. It integrates a cardiopulmonary audio encoder with a medical language model backbone, trained on
StethoBench — a comprehensive benchmark of 77,027 instruction–response pairs from 16,125 labeled recordings.
This work is published in the Transactions on Machine Learning Research (TMLR).
StethoLM connects a
COLA audio encoder (EfficientNet-based, pre-trained on cardiopulmonary sounds via
CaReAQA) to
MedGemma-4B-IT via a learned MLP prefix projector. The audio is encoded into a short sequence of prefix tokens that are prepended to the text input of the language model. All components — audio encoder, prefix projector, and language model (via LoRA) — are jointly fine-tuned end-to-end.
1git clone https://github.com/askyishan/StethoLM
2cd StethoLM
3pip install -r requirements.txt
1python predict.py \
2 --input_jsonl data/stethobench.jsonl \
3 --output_jsonl predictions.jsonl \
4 --audio_dir /path/to/audio_files \
5 --checkpoint checkpoints/stetholm_adapter.pt \
6 --model_name google/medgemma-4b-it \
7 --audio_encoder cola \
8 --split test
StethoLM was trained on
StethoBench. The training split comprises recordings from 7 in-domain datasets; 4 additional datasets are held out as out-of-distribution (OOD) test sets.
1@article{stetholm2025,
2 title = {StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks},
3 author = {Wang, Yishan and Wang, Tsai-Ning and Funk, Mathias and Saeed, Aaqib},
4 journal = {Transactions on Machine Learning Research},
5 year = {2026},
6 url = {https://huggingface.co/askyishan/StethoLM}
7}
