Tina (Tiny Reasoning Models via LoRA) models are all fine-tuned adapters on the base model
deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B.
This LoRA adapter in this repo is fine-tuned with the dataset
RUC-AIBOX/STILL-3-Preview-RL-Data.
Please refer to our paper
Tina: Tiny Reasoning Models via LoRA for more training details.
The Tina model is meant to be used in combination with the base model as a standard adapter. Particularly, we release all checkpoints we have for each Tina model and one could select different checkpoint to use by specifying the subfolder.
1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
3
4base_model = AutoModelForCausalLM.from_pretrained(
5 "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
6 device_map="auto"
7)
8tokenizer = AutoTokenizer.from_pretrained(
9 "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
10)
11
12model = PeftModel.from_pretrained(
13 base_model,
14 "Tina-Yi/R1-Distill-Qwen-1.5B-STILL",
15 subfolder="checkpoint-2000" # checkpoint 2000 is the best
16)
