Anonymous NeurIPS 2026 submission. Companion code: see the linked anonymized GitHub repository. Preprocessing pipeline, training code, and evaluation cookbook will be released upon acceptance.

Model Description

smb-v1 models the complex time-varying dynamics of cancer biology through structured clinical signals. It treats structured clinical data as a multimodal environment, fusing heterogeneous data streams into a unified patient state representation.

This checkpoint was trained using the SFT + JEPA curriculum objective on top of meta-llama/Meta-Llama-3-8B.

The model is designed to ingest and synthesize diverse structured modalities across the patient journey, including:

Temporal Physiological Signals: Modeling continuous longitudinal trajectories of laboratory values, vital signs, and functional status markers.
Clinical Events & Phenotypes: Encoding discrete, high-cardinality sequences of diagnosis codes (ICD), procedure events (CPT), and adverse events.
Therapeutic Interventions: Integrating complex treatment histories—including systemic therapies, radiation dosing schedules, and surgical interventions.
Molecular & Genomic Profiles: Embedding high-dimensional static and dynamic biomarker panels.
Oncologic Staging & Outcomes: Processing structured tumor staging (TNM), histology classifications, and survival endpoints.

Intended Use Cases

Predictive Risk Stratification: Forecasting adverse events, toxicity, or rapid progression based on historical trajectories.
Treatment Response Modeling: Simulating potential patient outcomes under different therapeutic regimens.
Patient Similarity Search: Identifying cohorts with similar biological and clinical progressions.
Clinical Trial Matching: Aligning complex patient states with structured eligibility criteria.

Usage

Inputs must be a pre-formatted patient timeline (the output of the preprocessing pipeline, which will be released upon acceptance). The companion code repository ships a synthetic example demonstrating the expected text format.

pip install transformers torch accelerate

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "anon-9421/smb-structure-llama3-8b-curriculum"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

input_text = open("examples/synthetic_patient.txt").read()
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
with torch.inference_mode():
    outputs = model(
        input_ids=inputs.input_ids,
        attention_mask=inputs.attention_mask,
        output_hidden_states=True,
        return_dict=True,
    )

patient_embedding = outputs.hidden_states[-1]
print(f"Patient representation shape: {patient_embedding.shape}")

Training Data

Single-institution longitudinal cancer EHR data containing structured information derived from pathology reports, clinical reports, biomarkers, and staging. The model was not trained on raw biomedical data of these modalities. Full data documentation will accompany the camera-ready release.

Citation

bibtex

@misc{smb_structure_anon_2026,
  title  = {The Patient is not a Moving Document: A World Model Training Paradigm for Longitudinal EHR},
  author = {Anonymous Authors},
  year   = {2026},
  note   = {NeurIPS 2026 submission. Under review.}
}