This repository contains the DeepSeek-R1-Distill-Llama-8B model fine-tuned for advanced medical reasoning tasks. The model has been converted to GGUF format for efficient inference with CPU and GPU backends, making it compatible with tools like llama.cpp and ollama.

Model Information

Base Model: unsloth/DeepSeek-R1-Distill-Llama-8B
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Training Dataset: Medical O1 Reasoning SFT
Quantization: Q8_0 GGUF

Fine-tuning Configuration

Epochs: 1
Max Steps: 60
Batch Size per Device: 2
Gradient Accumulation Steps: 4
Learning Rate: 2e-4
Optimizer: AdamW (8-bit)
Seed: 3407 (for reproducibility)

Usage

You can use this GGUF model with llama.cpp or ollama for inference.

Using `llama.cpp`

./main -m deepseek-medical-q8_0.gguf --temp 0.7 -p "A 61-year-old woman with involuntary urine loss during coughing but no leakage at night undergoes a gynecological exam and Q-tip test. What would cystometry reveal about residual volume and detrusor contractions?"

Using `ollama`

Move the GGUF model to the Ollama model directory:

mv deepseek-medical-q8_0.gguf ~/.ollama/models

Create a modelfile:

FROM deepseek-medical-q8_0.gguf

Run inference:

ollama run deepseek-medical-q8_0 "A 61-year-old woman with involuntary urine loss during coughing but no leakage at night undergoes a gynecological exam and Q-tip test. What would cystometry reveal about residual volume and detrusor contractions?"

Repository

Hugging Face Model Repository

NikkeS/DeepSeek-8B-Finetuned-Medical-O1-Reasoning-SFT-Q8_0-GGUF

DeepSeek R1 Medical Reasoning (GGUF Format)

Model Information

Fine-tuning Configuration

Usage

Using llama.cpp

Using ollama

Repository

NikkeS/
DeepSeek-8B-Finetuned-Medical-O1-Reasoning-SFT-Q8_0-GGUF

Using `llama.cpp`

Using `ollama`