med-llm-es: Spanish Medical Triage LLM

⚠️ MEDICAL DISCLAIMER - IMPORTANT

• ❌ This is NOT a medical device
• ❌ This does NOT provide medical advice
• ❌ Do NOT use for actual patient triage
• ❌ The model may produce incorrect, incomplete, or harmful outputs

• Always recommend professional medical consultation
• In real emergencies, call emergency services (112 in Europe, 911 in US)
• This model should only be used for learning about LLM fine-tuning
• Researchers: validate thoroughly before any downstream applications
• Deployers: assume full liability for any use cases

Project Status

What This Project Provides

Working Models

Training Datasets (Available)

• Distilled data: 5000+ examples from MiniMax-M2.5
• Preference data: 10K+ DPO training pairs
• Balanced data: Enhanced training sets

Technical Achievements

• Full RLHF pipeline (CPT → SFT → GRPO → DPO)
• Offline-capable quantized models
• Spanish medical language specialization
• Manchester Triage System (MTS) implementation

Use Cases (Educational)

1. Build domain-specific LLMs - Medical Spanish fine-tuning
2. Implement knowledge distillation - Using powerful teacher models
3. Apply RLHF techniques - GRPO and DPO for alignment
4. Optimize for edge deployment - GGUF quantization
5. Create safety-aligned models - Medical disclaimers and urgency levels

Pipeline

┌─────────────────────────────────────────────────────────────────────────┐
│  1. Data Prep      →  2. CPT       →  3. SFT       →  4. Distill    │
│  OpenMed + MTS      Spanish Med     Triage SFT      MiniMax-M2.5      │
│                                                                         │
│  5. GRPO        →  6. DPO       →  7. Quantize  →  8. Deploy       │
│  Rewards          Preference      GGUF Q5        Offline App         │
└─────────────────────────────────────────────────────────────────────────┘

Directory Structure

med-llm-es/
├── configs/
│   └── config.py              # Configuration settings
├── data/
│   ├── raw/                   # Downloaded OpenMed datasets
│   ├── translated/            # Spanish translations
│   ├── triage/                # Generated triage prompts
│   ├── distilled/             # Teacher-generated data (~10MB)
│   └── preference/            # DPO preference pairs (~10MB)
├── models/
│   ├── cpt-spanish-medical-v1/    # CPT model
│   ├── sft-spanish-triage-v1/     # SFT model
│   ├── grpo-spanish-triage-v1/    # GRPO model
│   ├── dpo-spanish-triage-v1/     # DPO model
│   └── gguf/                      # Quantized models (~2GB total)
├── scripts/
│   ├── 01_download_opendmed.py      # Download datasets
│   ├── 02_translate_to_spanish.py   # Translate to Spanish
│   ├── 03_generate_triage_data.py   # Create triage prompts
│   ├── 04_cpt_spanish_medical.py    # Continued Pre-Training
│   ├── 05_sft_triage.py             # Supervised Fine-Tuning
│   ├── 06_distillation_generate.py  # Knowledge Distillation
│   ├── 07_create_preference_data.py # Create DPO dataset
│   ├── 08_grpo_triage.py            # GRPO training
│   ├── 09_dpo_triage.py             # DPO training
│   ├── 10_quantize_gguf.py          # Quantization
│   └── 11_monitor_grpo.py           # Passive GRPO run monitor
├── checkpoints/               # Training checkpoints
├── reports/                   # Documentation
├── DEPLOYMENT_GUIDES.md      # Edge deployment instructions
└── README.md

Quick Start

Prerequisites

1. Google Colab Pro (for A100 GPU access) or local GPU (16GB+ VRAM)
2. MiniMax API Key (for distillation)
3. Google Drive (for storage)

Execution Order

1. Data Preparation
   bash

   1python scripts/01_download_opendmed.py 2python scripts/02_translate_to_spanish.py 3python scripts/03_generate_triage_data.py
2. Training (on Colab)
   bash

   1python scripts/04_cpt_spanish_medical.py # CPT 2python scripts/05_sft_triage.py # SFT 3python scripts/06_distillation_generate.py # Distillation 4python scripts/07_create_preference_data.py # Preference data 5python scripts/08_grpo_triage.py # GRPO 6python scripts/09_dpo_triage.py # DPO
3. Quantization
   python scripts/10_quantize_gguf.py

Triage System

Configuration

Deployment

• Android (Termux)
• iOS (MLX)
• Desktop
• Raspberry Pi

Cost Estimate

Limitations & Risks

1. Model may hallucinate - Incorrect medical information
2. Limited training data - Not comprehensive medical coverage
3. No clinical validation - Never tested in real settings
4. Language bias - Trained on specific Spanish variants
5. Quantization losses - Accuracy trade-offs from compression

License

• Base models: LFM2.5 (Liquid AI), MiniMax-M2.5 (MiniMax)
• Training: Apache 2.0 / TRL

Acknowledgments

• OpenMed - Medical datasets
• Liquid AI - LFM2.5 models
• MiniMax - Teacher model
• TRL - Training library
• Unsloth - Efficient fine-tuning