Gemma 12B Medical MC QLoRA v2
A medical multiple choice focused fine-tuned version of Google's Gemma 3 12B Instruct model using QLoRA (Quantized Low-Rank Adaptation) techniques, specifically trained on medical multiple choice datasets.
Model Details
Model Description
This model is a QLoRA adapter fine-tuned on Google's Gemma 3 12B Instruct model specifically for medical multiple choice applications. The model has been trained for 7000 steps on medical multiple choice datasets to understand and generate medical content in a multiple choice format while maintaining the conversational capabilities of the base model.
- Developed by: inarikami
- Model type: Causal Language Model (QLoRA Adapter)
- Language(s): English
- Base Model: google/gemma-3-12b-it
- License: Same as base model (Gemma License)
- Checkpoint: 7000 steps
Model Sources [optional]
- Repository: [More Information Needed]
- Paper [optional]: [More Information Needed]
- Demo [optional]: [More Information Needed]
Uses
Direct Use
This model is designed for medical question-answering, educational purposes, and healthcare-related conversations. It can be used to:
- Answer general medical questions
- Provide educational content about health topics
- Assist with medical terminology explanations
- Support healthcare professionals with information retrieval
Downstream Use
The model can be further fine-tuned for specific medical domains such as:
- Clinical documentation
- Medical summarization tasks
- Specialized medical subspecialties
- Integration into healthcare chatbots or educational platforms
Out-of-Scope Use
Important: This model should NOT be used for:
- Medical diagnosis or treatment recommendations
- Emergency medical situations
- Replacing professional medical advice
- Making clinical decisions without human oversight
- Patient care without qualified medical supervision
Bias, Risks, and Limitations
This model has several important limitations:
- Medical Accuracy: While trained on medical data, the model may generate inaccurate or outdated medical information
- Training Data Bias: The model may reflect biases present in medical literature and training data
- Language Limitations: Primarily trained on English medical content
- Hallucination Risk: Like all large language models, it may generate plausible-sounding but incorrect information
- Regional Variations: Medical practices and guidelines vary by region; the model may not reflect local standards
Recommendations
Critical Safety Guidelines:
- Always verify medical information with qualified healthcare professionals
- Use only as a supplementary educational tool, never for medical decision-making
- Implement human oversight for any healthcare-related applications
- Regularly update and validate outputs against current medical guidelines
- Be aware of potential biases in medical recommendations across different populations
How to Get Started with the Model
1from peft import PeftModel
2from transformers import AutoTokenizer, AutoModelForCausalLM
3import torch
4
5# Load base model and tokenizer
6base_model = "google/gemma-3-12b-it"
7model = AutoModelForCausalLM.from_pretrained(
8 base_model,
9 torch_dtype=torch.float16,
10 device_map="auto"
11)
12tokenizer = AutoTokenizer.from_pretrained(base_model)
13
14# Load QLoRA adapter
15model = PeftModel.from_pretrained(model, "your-username/gemma-12b-medical-qlora-v2")
16
17# Generate medical responses
18prompt = "What are the symptoms of diabetes?"
19inputs = tokenizer(prompt, return_tensors="pt")
20with torch.no_grad():
21 outputs = model.generate(**inputs, max_length=200, temperature=0.7)
22response = tokenizer.decode(outputs[0], skip_special_tokens=True)
23print(response)
Training Details
Training Data
This model was fine-tuned on a medical multiple choice question dataset. The training data consists of:
- Medical multiple choice questions and answers
- Healthcare-related Q&A pairs in multiple choice format
- Medical terminology and concept explanations structured as multiple choice problems
Note: The model's training on multiple choice data means it may be particularly well-suited for medical education scenarios involving multiple choice questions, but may require additional fine-tuning for other medical text generation tasks.
Training Procedure
Preprocessing [optional]
[More Information Needed]
Training Hyperparameters
- Training regime: [More Information Needed]
Speeds, Sizes, Times [optional]
[More Information Needed]
Evaluation
Testing Data, Factors & Metrics
Testing Data
[More Information Needed]
Factors
[More Information Needed]
Metrics
[More Information Needed]
Results
[More Information Needed]
Summary
Model Examination [optional]
[More Information Needed]
Environmental Impact
Carbon emissions can be estimated using the
Machine Learning Impact calculator presented in
Lacoste et al. (2019).
- Hardware Type: [More Information Needed]
- Hours used: [More Information Needed]
- Cloud Provider: [More Information Needed]
- Compute Region: [More Information Needed]
- Carbon Emitted: [More Information Needed]
Technical Specifications [optional]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
[More Information Needed]
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
[More Information Needed]
Framework versions
