mkurman/Llama-3.2-MedIT-3B-R1 is a fine-tuned variant of meta-llama/Llama-3.2-3B-Instruct, adapted specifically for exploring natural language understanding and reasoning. This model leverages a multi-stage training approach, combining Blurred Thoughts Supervised Fine-Tuning (BT-SFT) and Group Relative Policy Optimization (GRPO) with an LLM evaluator to enhance its performance on specialized tasks.
-
Initial Blurred Thoughts Supervised Fine-Tuning (BT-SFT):
- Base Model: meta-llama/Llama-3.2-3B-Instruct
- Parameters: 2000 steps, batch size 2, accumulation iterations 16, learning rate 1e-6
- Dataset: open-thoughts/OpenThoughts-114k
- Details: For further information on BT-SFT, see the detailed post and the GitHub repository.
-
Group Relative Policy Optimization (GRPO) Stage 1:
- Dataset: FreedomIntelligence/medical-o1-verifiable-problem
- Training: 200 steps
- LLM Evaluator mkurman/Qwen2.5-14B-DeepSeek-R1-1M
- Details: For further information on GRPO with LLM evaluators, see the GitHub repository.
-
Group Relative Policy Optimization (GRPO) Stage 2:
- Dataset: open-r1/OpenR1-Math-220k
- Training: 200 steps
- LLM Evaluator deepseek/deepseek-r1-distill-qwen-14b (OpenRouterAI)
-
open-thoughts/OpenThoughts-114k:
A dataset consisting of open-ended thoughts that supports diverse conversational contexts during the initial supervised fine-tuning.
-
FreedomIntelligence/medical-o1-verifiable-problem:
A dataset curated for enhancing the model's capabilities in addressing verifiable medical problems.
-
open-r1/OpenR1-Math-220k:
A dataset designed to improve the model's reasoning and problem-solving skills in mathematical contexts.
-
Research and Experimental Applications:
This model is optimized for academic research and exploratory projects. It is ideal for investigating advanced fine-tuning methods and evaluating performance on task-oriented conversational scenarios.
-
Controlled Environments:
Users should deploy this model only within controlled experimental frameworks where rigorous evaluation and proper safety guardrails are in place.
-
Not for Clinical or Production Use:
The model’s outputs have not been validated for clinical accuracy or professional decision-making. It must not be used as a primary source for medical, legal, or safety-critical information.
-
Safety and Guardrails:
All users must implement appropriate safety measures and validation protocols. The model may produce biased or inaccurate results and should be used with caution.
-
Experimental Nature:
Given its research-oriented design, the model’s performance can vary widely based on input and context. It is essential to perform thorough testing and validation before drawing any conclusions from its outputs.
This model is released under the Llama 3.2 license. Users must adhere to the terms specified in the license when utilizing this model.
