You can deploy this model and start earning money today!

RezaAskari/
GPT2-optimal_insulin_calculator

Name: GPT2-optimal_insulin_calculator
Availability: InStock

safetensors

2106.09685

2305.14314

Views

No views yet

GPT-2 Insulin Titration Fine-Tuning

A specialized fine-tuning project for GPT-2 to provide insulin dosage adjustment recommendations based on 7-day fasting blood glucose (FBG) measurements.

🎯 Project Overview

This project fine-tunes GPT-2 language models to analyze patient blood glucose patterns and recommend insulin dosage adjustments. The system implements clinical decision rules for insulin titration based on average FBG levels and variability patterns.

Decision Logic

High FBG (>180 mg/dL): Increase insulin by 20 units
Low FBG (<100 mg/dL): Decrease insulin by 20 units
Optimal FBG (100-180 mg/dL): No change in dosage
High variability (std >40): Consider smaller adjustments

📁 Project Structure

fine_tuning/
├── fine_tune.py              # Standard GPT-2 fine-tuning
├── fine_tune_lora.py         # LoRA (Low-Rank Adaptation) fine-tuning
├── fine_tune_qlora.py        # QLoRA (Quantized LoRA) fine-tuning
├── push_to_hub.py            # Script to upload models to HuggingFace Hub
├── gpt2_insulin_titration/   # Standard fine-tuned model
├── gpt2_insulin_lora/        # LoRA fine-tuned model
└── gpt2_insulin_qlora/       # QLoRA fine-tuned model

🚀 Quick Start

Prerequisites

pip install torch transformers datasets peft accelerate bitsandbytes huggingface-hub

Basic Fine-Tuning

Run standard GPT-2 fine-tuning:

python fine_tune.py

Advanced Fine-Tuning Options

LoRA Fine-Tuning (Memory Efficient)

python fine_tune_lora.py

QLoRA Fine-Tuning (Most Memory Efficient)

python fine_tune_qlora.py

💻 System Requirements

Hardware

Minimum: 8GB RAM (CPU training)
Recommended: NVIDIA GPU with 8GB+ VRAM
Supported: Apple Silicon (M1/M2/M3) with MPS acceleration

Software

Python 3.8+
PyTorch 2.0+
Transformers 4.30+
CUDA 11.8+ (for NVIDIA GPUs)

🔧 Configuration

Training Parameters

Parameter	Standard	LoRA	QLoRA
Epochs	5	3	3
Batch Size	4	8	8
Learning Rate	5e-5	3e-4	2e-4
Warmup Steps	50	50	50
Max Length	128	128	128

LoRA Configuration

Rank (r): 16
Alpha: 32
Dropout: 0.1
Target Modules: c_attn, c_proj

QLoRA Configuration

4-bit quantization
Double quantization enabled
NF4 quantization type
FP16 compute dtype

📊 Dataset

The project generates synthetic training data with 100 samples covering four scenarios:

High Glucose: FBG consistently >180 mg/dL
Low Glucose: FBG consistently <100 mg/dL
Optimal Range: FBG between 100-180 mg/dL
Variable: Mixed high and low readings

Each training example follows the format:

Patient's fasting blood glucose over 7 days (mg/dL): [values]. 
Insulin adjustment recommendation: [INCREASE/DECREASE/NO CHANGE]

🧪 Testing

The fine-tuned models are tested with three scenarios:

High FBG pattern (requires increase)
Low FBG pattern (requires decrease)
Optimal FBG pattern (no change needed)

📤 Deployment

Local Usage

python

from transformers import GPT2LMHeadModel, GPT2Tokenizer

# Load model
model = GPT2LMHeadModel.from_pretrained("./gpt2_insulin_titration")
tokenizer = GPT2Tokenizer.from_pretrained("./gpt2_insulin_titration")

# Make prediction
fbg_values = "210, 195, 220, 205, 190, 215, 200"
prompt = f"Patient's fasting blood glucose over 7 days (mg/dL): {fbg_values}. Insulin adjustment recommendation:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)
recommendation = tokenizer.decode(outputs[0], skip_special_tokens=True)

HuggingFace Hub

Push your model to HuggingFace Hub:

bash

# Login to HuggingFace
huggingface-cli login

# Push model
python push_to_hub.py

Or use the model directly from the Hub:

model = GPT2LMHeadModel.from_pretrained("MohammadReza99/GPT2-optimal_insulin_calculator")

📈 Performance Comparison

Method	Training Time	Memory Usage	Model Size	Accuracy
Standard	~10 min	4-6 GB	548 MB	Baseline
LoRA	~5 min	2-3 GB	3.2 MB (adapter)	~95% of baseline
QLoRA	~7 min	1-2 GB	3.2 MB (adapter)	~93% of baseline

⚠️ Important Notes

Medical Disclaimer

This is a demonstration project for educational purposes. The model should NOT be used for actual medical decisions without proper validation and regulatory approval.

Limitations

Synthetic training data may not capture all real-world scenarios
Model outputs should be validated by healthcare professionals
Does not account for individual patient factors beyond FBG

🔬 Future Improvements

Incorporate additional patient factors (weight, age, insulin type)
Add hypoglycemia risk assessment
Implement continuous glucose monitoring (CGM) data support
Expand training data with real clinical cases
Add uncertainty quantification for recommendations
Implement multi-language support

📚 References

📝 License

This project is for educational and research purposes. Please consult appropriate medical guidelines and regulations before any clinical application.

🤝 Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues for improvements.

📧 Contact

For questions or collaboration opportunities, please reach out through the HuggingFace Hub repository: MohammadReza99/GPT2-optimal_insulin_calculator

RezaAskari/GPT2-optimal_insulin_calculator