Views
No views yet
deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B.1from peft import PeftModel
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4base_model_name = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
5adapter_repo = "DashMav/deepseek-r1-lora-adapter"
6
7# Load base model
8base_model = AutoModelForCausalLM.from_pretrained(base_model_name)
9# Load LoRA adapter
10model = PeftModel.from_pretrained(base_model, adapter_repo)
11
12# Load tokenizer
13tokenizer = AutoTokenizer.from_pretrained(base_model_name)
14
15✨ Example Generation
16inputs = tokenizer("Write a poem about AI:", return_tensors="pt").to(model.device)
17outputs = model.generate(**inputs, max_new_tokens=100)
18print(tokenizer.decode(outputs[0], skip_special_tokens=True))
19
20
21⚡️ Details
22| Item | Value |
23| -------------------- | ------------------------------------------- |
24| **Base model** | `deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B` |
25| **Adapter type** | LoRA (Low-Rank Adaptation) |
26| **Train epochs** | 10 |
27| **Trainable params** | \~1 Million (\~0.06% of total) |
28| **Task** | Causal Language Modeling |
29
30✅ Notes
31
32This repo only contains the LoRA weights (adapter_model.safetensors).
33You need to load it with the base model for generation.
34
35LoRA reduces compute cost for fine-tuning and enables lightweight domain adaptation.
36
37📜 License
38
39This adapter inherits the license of the base model.
40Please refer to DeepSeek R1 License.
41
42📣 Author
43Adapter fine-tuned by DashMav 🚀
44
45
46
47✨ Enjoy and happy prompting!