ThaiLLM-27B-Prescreen
ThaiLLM-27B-Prescreen is a reinforcement learning fine-tuned version of google/medgemma-27b-text-it, trained specifically for patient pre-screening. Given a patient profile and current symptoms, the model predicts the likely disease, recommends the appropriate hospital department, and estimates clinical severity.
Training Details
The model was trained using Prime-Intellect's
prime-rl framework
Data
The model was trained using
https://huggingface.co/datasets/ThaiLLM/med-prescreen dataset with Prime Intellect's verifier framework.
Training Configuration
This was the prime-rl configuration used to train the model
1max_steps = 500
2seq_len = 16384
3
4[deployment]
5type = "single_node"
6num_train_gpus = 2
7num_infer_gpus = 6
8
9[inference.parallel]
10dp = 6
11
12[trainer.model]
13attn = "flash_attention_3"
14optimization_dtype = "bfloat16"
15reduce_dtype = "bfloat16"
16
17[trainer.model.lora]
18rank = 64
19alpha = 128
20
21[trainer.model.ac]
22
23[trainer.optim]
24lr = 5e-5
25
26[orchestrator]
27batch_size = 512
28rollouts_per_example = 16
29num_train_workers = 2
30
31[orchestrator.wandb.log_extras]
32samples = true
33interval = 1
34
35[orchestrator.sampling]
36max_tokens = 8192
37
38[[orchestrator.env]]
39id = "prescreen_classification"
40name = "prescreen_classification"
41
42[ckpt]
43interval = 50
44keep_interval = 50
Reward Functions
The environment was developed following the verifiers framework with the following reward functions with the following weights for each reward [2.0, 1.0, 1.0, 0.3]
1async def disease_reward(completion, answer):
2 response = completion[-1]["content"]
3 predicted = _extract_tag(response, "disease")
4 if predicted is None:
5 return 0.0
6 predicted = predicted.lower()
7 true_disease = answer.get("disease", "").lower()
8 if predicted == true_disease:
9 return 1.0
10 return 0.0
11
12async def department_reward(completion, answer):
13 response = completion[-1]["content"]
14 predicted = _extract_tag(response, "department").lower()
15 if predicted is None:
16 return 0.0
17 answer = answer.get("department", "").lower()
18 return 1.0 if predicted == answer else 0.0
19
20async def severity_reward(completion, answer):
21 response = completion[-1]["content"]
22 predicted = _extract_tag(response, "severity").lower()
23 if predicted is None:
24 return 0.0
25 answer = answer.get("severity", "").lower()
26 return 1.0 if predicted == answer else 0.0
27
28async def format_reward(completion, answer) -> float:
29 response = completion[-1]["content"]
30 text_without_think = re.sub(r"<unused94>.*?</unused94>", "", response, flags=re.DOTALL | re.IGNORECASE) # medgemma uses the the <unused94> token instead of <think>
31 tags = ["disease", "department", "severity"]
32 present = sum(1 for t in tags if f"<{t}>" in text_without_think.lower() and f"</{t}>" in text_without_think.lower())
33 return present / len(tags)
Performance
We benchmark against four baselines spanning general-purpose reasoning models (Qwen3-30B-A3B-Thinking-2507, Qwen3-8B) and medical-domain models (medgemma-27b-text-it, medgemma1.5-4b-it). ThaiLLM-27B-Prescreen improves disease F1 by +0.448 over its base model (0.287 → 0.735) and outperforms Qwen3-30B-A3B-Thinking-2507 at 0.515. Department routing also improves meaningfully (+0.048 F1 over the base, +0.077 over Qwen3-30B-A3B-Thinking-2507), with the largest gain appearing in accuracy (0.436 → 0.677), suggesting the model is substantially better at picking the single correct department rather than hedging across plausible ones. There is however a severity trade-off, severity F1 is slightly below the base MedGemma-27B (0.571 vs 0.601) and noticeably below Qwen3-30B-A3B-Thinking (0.659). However, ThaiLLM-27B-Prescreen achieves the highest severity accuracy of any model tested (0.799), and the per-class breakdown below shows why the two metrics diverge: the model is strong on the two clinically consequential classes (Emergency and Visit Hospital / Clinic) and fails entirely on Observe at Home.
Overall Performance (F1)
| Model | Disease | Department | Severity |
|---|
| Qwen3-30B-A3B-Thinking-2507 | 0.515 | 0.464 | 0.659 |
| Qwen3-8B | 0.157 | 0.449 | 0.574 |
| medgemma1.5-4b-it | 0.095 | 0.424 | 0.525 |
| medgemma-27b-text-it | 0.287 | 0.493 | 0.601 |
| ThaiLLM-27B-Prescreen | 0.735 | 0.541 | 0.571 |
Disease Classification
| Model | F1 | Precision | Recall | Accuracy |
|---|
| Qwen3-30B-A3B-Thinking-2507 | 0.515 | 0.562 | 0.509 | 0.510 |
| Qwen3-8B | 0.157 | 0.215 | 0.148 | 0.149 |
| medgemma1.5-4b-it | 0.095 | 0.131 | 0.082 | 0.076 |
| medgemma-27b-text-it | 0.287 | 0.336 | 0.266 | 0.286 |
| ThaiLLM-27B-Prescreen | 0.735 | 0.776 | 0.730 | 0.729 |
Department Classification
| Model | F1 | Precision | Recall | Accuracy |
|---|
| Qwen3-30B-A3B-Thinking-2507 | 0.464 | 0.466 | 0.677 | 0.420 |
| Qwen3-8B | 0.449 | 0.419 | 0.648 | 0.358 |
| medgemma1.5-4b-it | 0.424 | 0.394 | 0.541 | 0.358 |
| medgemma-27b-text-it | 0.493 | 0.469 | 0.678 | 0.436 |
| ThaiLLM-27B-Prescreen | 0.541 | 0.606 | 0.518 | 0.677 |
Severity Classification
| Model | F1 | Precision | Recall | Accuracy |
|---|
| Qwen3-30B-A3B-Thinking-2507 | 0.659 | 0.722 | 0.639 | 0.774 |
| Qwen3-8B | 0.574 | 0.858 | 0.601 | 0.771 |
| medgemma1.5-4b-it | 0.525 | 0.616 | 0.529 | 0.715 |
| medgemma-27b-text-it | 0.601 | 0.835 | 0.609 | 0.755 |
| ThaiLLM-27B-Prescreen | 0.571 | 0.548 | 0.599 | 0.799 |
| Class | Precision | Recall | F1 | Support |
|---|
| Emergency | 0.878 | 0.857 | 0.867 | 84 |
| Observe At Home | 0.000 | 0.000 | 0.000 | 36 |
| Visit Hospital / Clinic | 0.767 | 0.940 | 0.845 | 168 |
The model never predicts Observe at Home — those 36 cases are being absorbed into Visit Hospital / Clinic instead. The collapse of the Observe at Home class is a real limitation of the system and should be taken into account when deploying the model.
Usage
vLLM
1uv run --with vllm vllm serve google/medgemma-27b-text-it \
2 --enable-lora \
3 --lora-modules prescreen=ThaiLLM/ThaiLLM-27B-Prescreen \
4 --max-lora-rank 64
