Model Card for Model ID
Better tuned deepseek-r1 model using dpo and specific customer service dataset
Model Details
Model Description
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- Developed by: [More Information Needed]
- Funded by [optional]: [More Information Needed]
- Shared by [optional]: [More Information Needed]
- Model type: [More Information Needed]
- Language(s) (NLP): [More Information Needed]
- License: [More Information Needed]
- Finetuned from model [optional]: [More Information Needed]
Model Sources [optional]
- Repository: [More Information Needed]
- Paper [optional]: [More Information Needed]
- Demo [optional]: [More Information Needed]
How to use this model
first we need unsloth
Normally using pip install unsloth is enough
Temporarily as of Jan 31st 2025, Colab has some issues with Pytorch
Using pip install unsloth will take 3 minutes, whilst the below takes <1 minute:
%%capture
!pip install --no-deps bitsandbytes accelerate xformers==0.0.29 peft trl triton
!pip install --no-deps cut_cross_entropy unsloth_zoo
!pip install sentencepiece protobuf datasets huggingface_hub hf_transfer
!pip install --no-deps unsloth
from unsloth import FastLanguageModel
from transformers import AutoTokenizer
Path to your fine-tuned model
model_path = "drive/MyDrive/deepseek-r1-reasoning-dpo" # Replace
Load the base model optimized with Unsloth
model, tokenizer = FastLanguageModel.from_pretrained(
model_name=model_path,
max_seq_length=4096, # Adjust based on model capability
dtype=torch.float16,
load_in_4bit=True, # Enable quantization for efficiency
)
Optimize LoRA model for inference (2x faster with Unsloth)
FastLanguageModel.for_inference(model)
Move model to GPU if available
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
print("Model loaded successfully!")
---------------------------------------------
