Base: google/gemma-4-e4b
Method: LoRA (rank 64, alpha 128, BF16) + DPO (beta 0.1)
Strategy: DDP on 8x A100 80GB (data parallelism)
Data: LumkoMeds gold v2 (41,279 SFT + 14,000 DPO examples)
CoT: Chain-of-thought reasoning with blocks
Tools: sep_lookup, ddi_check, dosing_calculator, allergy_cross_check, formulary_check
Safety: Refusal/boundary training curriculum
Custom tokens: 11 non-splittable tokens for tool calls and clinical annotations
Chat template: Custom with 'tool' role support
Max sequence length: 8192

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("path/to/merged_model")
tokenizer = AutoTokenizer.from_pretrained("path/to/merged_model")

LeroyNgoma/lumkomeds-gemma4-e4b

LumkoMeds Clinical SLM (Gemma-4-E4B Fine-tuned)

Training

Usage

LeroyNgoma/
lumkomeds-gemma4-e4b