⚠️ Status: Weights not uploaded yet. Training is staged but blocked on HF Jobs credit top-up. Add credits at https://huggingface.co/settings/billing and then ask the ML-Intern agent to launch.

Planned: full supervised fine-tune of Qwen/Qwen2.5-7B-Instruct on the FreedomIntelligence/medical-o1-reasoning-SFT (en split, 19.7K reasoning-style medical QA examples).

TL;DR — Where to look first

train_medical_qwen.py — single-file SFT script with embedded justifications for every hyperparameter.
The "How to launch" section below — three copy-pasteable commands (smoke, full, eval).
The "Expected MedQA accuracy" table — sanity-check targets.

Recipe (grounded in published results)

Based on HuatuoGPT-o1 (arXiv:2412.18925, 107 upvotes on HF papers). HuatuoGPT-o1 uses this dataset for its SFT stage and the HuggingFace TRL documentation uses the identical preprocessing block as its canonical example.

Preprocessing (verbatim from TRL docs)

python

def preprocess_function(example):
    return {
        "prompt": [{"role": "user", "content": example["Question"]}],
        "completion": [{
            "role": "assistant",
            "content": f"<think>{example['Complex_CoT']}</think>{example['Response']}",
        }],
    }

Hyperparameters

Setting	Value
Base model	`Qwen/Qwen2.5-7B-Instruct` (7.6B params)
Method	Full supervised fine-tune (no LoRA)
Epochs	3
Effective batch size	32 (per_device 2 × grad_accum 16)
Learning rate	5e-6
LR scheduler	cosine, warmup_ratio 0.1
Max sequence length	4096
Precision	bf16
Optimizer	`paged_adamw_8bit` (bitsandbytes)
Gradient checkpointing	✓ (`use_reentrant=False`)
Attention	flash-attn 2 via `kernels-community/flash-attn2`
Loss	completion-only (default for prompt-completion datasets)

Hardware & timing

Item	Value
GPU	a100-large (1× 80 GB A100)
Wall-time estimate	~4–6 h (≈ 1.85K steps at ~10 s/step)
Cost estimate	≈ $16–24 at $4/h

VRAM budget (7B full FT on a single 80 GB A100):

Weights bf16 ≈ 15 GB
Gradients bf16 ≈ 15 GB
paged_adamw_8bit states ≈ 15 GB
Activations (seq 4096, batch 2, grad ckpt) ≈ 10–15 GB
Total ≈ 55–60 GB — comfortable margin.

Monitoring

Trackio space: akshaypulusu/ml-intern-medqao1s (auto-seeded on launch)
Project: qwen-medical-o1-sft
Run: qwen2.5-7b-medical-o1-sft_lr5e-6_bs32_ep3
Alert callback fires on divergence, NaN loss, high initial loss, milestone (loss < 0.5), and training completion.

Files in this repo

File	Purpose
`train_medical_qwen.py`	SFT training. Supports `MEDICAL_SMOKE=1` for a 4-step preflight run.
`eval_medqa.py`	Evaluate trained model on MedQA-USMLE (4-option) test split (1.3K Q). Logs accuracy to trackio and writes `eval_out/summary.json`.
`inference_example.py`	Minimal load-and-chat demo.
`requirements.txt`	Pinned dependencies for reproducibility outside HF Jobs.
`README.md`	This file.

How to launch (once credits are available)

Smoke test (preflight, ~$1, ~10 min)

python

hf_jobs.run(
    script="train_medical_qwen.py",
    dependencies=["transformers", "trl", "datasets", "accelerate", "trackio", "kernels", "bitsandbytes", "torch"],
    hardware_flavor="a100-large",
    timeout="45m",
    env={"MEDICAL_SMOKE": "1"},
    trackio_space_id="akshaypulusu/ml-intern-medqao1s",
    trackio_project="qwen-medical-o1-sft",
)

Full training (~$16–24, ~4–6 h)

Same call, drop MEDICAL_SMOKE, raise timeout to "8h".

Evaluation (after training)

python

hf_jobs.run(
    script="eval_medqa.py",
    dependencies=["transformers", "datasets", "trackio", "kernels", "torch"],
    hardware_flavor="a10g-large",
    timeout="3h",
    trackio_space_id="akshaypulusu/ml-intern-medqao1s",
    trackio_project="qwen-medical-o1-sft",
)

Expected MedQA accuracy

For reference, on the MedQA-USMLE 4-option test set:

Model	Acc	Source
Qwen2.5-7B-Instruct (base, no FT)	~50–55%	typical zero-shot CoT
HuatuoGPT-o1-8B (Llama base, same dataset)	~72%	arXiv:2412.18925
Med42 (Llama-2-70B + medical FT)	~72% USMLE	arXiv:2404.14779
MEDITRON-70B	~70%	arXiv:2311.16079

Our 7B SFT run should land roughly in the 65–72% range — above the base model, in the same ballpark as the larger published baselines.

Alternative dataset to consider for follow-up

If results plateau, swap in UCSC-VLAA/MedReason (32.6K KG-grounded CoT, arXiv:2504.00993). MedReason-8B reportedly beats HuatuoGPT-o1-8B by up to 4.2% on MedBullets. Schema is different (question / answer / reasoning / options) so preprocessing must be adapted.

Limitations and risks

Not for clinical use. This model is research-only. Medical decisions must be made by a licensed clinician.
Hallucination. All LLMs invent plausible-sounding but incorrect medical facts. CoT reasoning does not eliminate this — it may make wrong answers more persuasive.
Dataset bias. medical-o1-reasoning-SFT is distilled from a teacher LLM on verifiable MCQ-style problems. It does not cover patient interaction, multi-turn clinical dialogue, or rare-disease presentations.
English only. The en config is used; performance on other languages is not optimized for.
Evaluation scope. MedQA accuracy is one signal; it does not measure safety, calibration, refusal behaviour, or robustness to adversarial prompts.

Citations

Primary recipe and dataset:

bibtex

@article{huatuogpt-o1,
  title   = {HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs},
  author  = {Chen, Junying and Cai, Zhenyang and Ji, Ke and Wang, Xidong and Liu, Wanlong and Wang, Rongsheng and Hou, Jianye and Wang, Benyou},
  journal = {arXiv preprint arXiv:2412.18925},
  year    = {2024},
  url     = {https://arxiv.org/abs/2412.18925}
}

Base model:

bibtex

@article{qwen2.5,
  title   = {Qwen2.5 Technical Report},
  author  = {Qwen Team},
  journal = {arXiv preprint arXiv:2412.15115},
  year    = {2024}
}

akshaypulusu/Qwen2.5-7B-Medical-o1-SFT

Qwen2.5-7B-Medical-o1-SFT

TL;DR — Where to look first

Recipe (grounded in published results)

Preprocessing (verbatim from TRL docs)

Hyperparameters

Hardware & timing

Monitoring

Files in this repo

How to launch (once credits are available)

Smoke test (preflight, ~$1, ~10 min)

Full training (~$16–24, ~4–6 h)

Evaluation (after training)

Expected MedQA accuracy

Alternative dataset to consider for follow-up

Limitations and risks

Citations

akshaypulusu/
Qwen2.5-7B-Medical-o1-SFT