Model Description
SmolLM2-Medical-LoRA is a fine-tuned version of SmolLM2-135M, optimized for short medical Q&A tasks.
The model was trained using LoRA (Low-Rank Adaptation) for efficient and fast training on Google Colab.
Typical tasks:
Answering medical questions
Explaining symptoms
Providing general health information
Simplifying complex medical topics
Dataset
The dataset consists of instruction–response pairs created specifically for this project.
Dataset Format (JSONL)
Each sample follows:
{
"text": "
\n"
}Training details
Base Model: SmolLM2-135M
Method: LoRA fine-tuning
Epochs: 7
Learning Rate: 2e-4
Batch Size: 2 (with gradient accumulation)
Max Length: 256 tokens
GPU: Google Colab T4
Frameworks: HuggingFace Transformers, Accelerate, PEFT
How to Use this model
Load with Transformers:
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("OmarrKhaledd/smollm2-medical-lora")
tokenizer = AutoTokenizer.from_pretrained("OmarrKhaledd/smollm2-medical-lora")
prompt = "What could cause chest pain and shortness of breath?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Example Output
User:
“What causes pain when swallowing?”
Model:
“Pain when swallowing (odynophagia) can happen due to throat infections, acid reflux, tonsillitis, or irritation of the esophagus…”
Evaluation
Model generates coherent, short medical explanations
Useful for educational Q&A
Safe: avoids giving diagnosis or prescriptions
Limitation
Not a real medical assistant
Limited dataset (small educational fine-tune)
May produce inaccurate or incomplete explanations
License
This model is released for research and educational use only.
Not intended for clinical, diagnostic, or therapeutic applications..
