LlamaTron RS1 Nemesis is a medical reasoning model produced by fine-tuning meta-llama/Llama-3.2-1B-Instruct on the Medical-Reasoning-SFT-MiniMax-M2.1 dataset using QLoRA. The dataset contains 204,773 clinical reasoning conversations with full chain-of-thought traces covering differential diagnosis, treatment planning, pharmacology, and clinical case analysis.
Despite being a 1 billion parameter model, it handles complex clinical questions with structured and coherent reasoning.
Loss decreased consistently across all steps with train and validation loss tracking closely. No overfitting observed.
Trained on
Medical-Reasoning-SFT-MiniMax-M2.1 released by
Maziyar Panahi under the OpenMed initiative.
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
3
4model_id = "Rumiii/LlamaTron_RS1_Nemesis_1B"
5
6tokenizer = AutoTokenizer.from_pretrained(model_id)
7model = AutoModelForCausalLM.from_pretrained(
8 model_id,
9 torch_dtype=torch.bfloat16,
10 device_map="auto",
11)
12
13pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
14
15messages = [
16 {
17 "role": "system",
18 "content": "You are LlamaTron RS1 Nemesis, a knowledgeable and compassionate medical AI assistant. Provide accurate, evidence-based medical information clearly and helpfully."
19 },
20 {
21 "role": "user",
22 "content": "What are the early symptoms of Type 2 Diabetes?"
23 },
24]
25
26output = pipe(
27 messages,
28 max_new_tokens=400,
29 do_sample=True,
30 temperature=0.7,
31 top_p=0.9,
32)
33
34print(output[0]["generated_text"][-1]["content"])
The full training code, merging scripts, and inference interface are available on GitHub:
github.com/sufirumii/LlamaTron-RS1-Nemesis-1B
