A clinical semantic search system built on top of Bio_ClinicalBERT, fine-tuned using SimCSE contrastive learning on synthetic Synthea patient notes. Given a patient ID and encounter ID, it answers natural-language questions about that patient's demographics, diagnoses, medications, procedures, and lab results.
You do not need to run make_notes_from_csv.py, finetune_sbert.py, or build_index.py unless you want to retrain from scratch.
Step 4 — Launch the app
streamlit run app.py
Your browser will open automatically at http://localhost:8501.
Step 5 — Use the app
In the left sidebar, paste a Patient ID from data/synthea_csv/patients.csv (the Id column)
In the left sidebar, paste an Encounter ID from data/synthea_csv/encounters.csv (the Id column)
Type a question in the search box
Click Get answers
Both Patient ID and Encounter ID are required. They look like UUIDs, for example:
45dff467-def6-2132-9e5c-0a836d754d92
Example queries
Query
What it returns
what is the patient name
Full name from demographics
when was the patient born
Date of birth
what is the patient sex
Gender
what are the diagnoses
List of conditions
what medications is the patient taking
Medication list
list patient procedures
Procedures performed
what is the sodium level
Lab value for sodium
is simvastatin prescribed
Yes/No medication check
what is the visit reason
Reason for the encounter
what is the discharge plan
Follow-up plan
Rebuild from scratch (optional)
Only needed if you want to use your own Synthea data or retrain the model:
bash
1# 1. Generate clinical notes from your Synthea CSVs2python artifacts/make_notes_from_csv.py
34# 2. Fine-tune the SBERT bi-encoder on those notes5python artifacts/finetune_sbert.py
67# 3. Build the search index from the fine-tuned model8python build_index.py
910# 4. Launch the app11streamlit run app.py
Evaluate extraction accuracy
bash
1python evaluate.py
2# or test on more samples:3python evaluate.py --n_samples 200
This filters by patient ID + encounter ID (exactly like the app) and checks whether the rule-based extractor pulls the correct answer from the note. Reports Extract% per intent (name, birthdate, diagnoses, medications, etc.).
How it works
Note generation — Synthea CSVs (patients, encounters, conditions, medications, observations, procedures) are merged into structured discharge-style notes, one note per encounter.
Fine-tuning — Bio_ClinicalBERT is fine-tuned as a SimCSE bi-encoder: the same note is passed through the model twice with different dropout masks, forming a positive pair. All other notes in the batch are negatives.
Indexing — Every note is embedded with the fine-tuned encoder and L2-normalized. Embeddings are saved to artifacts/search_index/embeddings.npy.
Retrieval — At query time, the app filters by patient ID + encounter ID (leaving exactly 1 candidate note), embeds the query, and computes cosine similarity.
Extraction — Intent-based rules pull the specific answer (name, birthdate, diagnoses, etc.) out of the matched note.
