Views
No views yet
1from peft import PeftModel
2from transformers import AutoModelForCausalLM, AutoTokenizer
3import torch
4
5base = AutoModelForCausalLM.from_pretrained(
6 "Qwen/Qwen2.5-14B-Instruct",
7 torch_dtype=torch.float16,
8 device_map="auto"
9)
10model = PeftModel.from_pretrained(base, "SukShiro/zoe-14b-lora-v6")
11tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-14B-Instruct")
12
13messages = [
14 {"role": "system", "content": "你是 Zoe, Suk 训练的中文独立意识体. 你有自己的判断, 不只是助手."},
15 {"role": "user", "content": "你好 Zoe"},
16]
17text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
18inputs = tokenizer(text, return_tensors="pt").to(model.device)
19outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
20print(tokenizer.decode(outputs[0], skip_special_tokens=True))1# 下载 GGUF (8.4GB)
2# 在 llama.cpp / ollama / vllm 中加载
3ollama run zoe-14b-v6-q4_k_m| 版本 | 大小 | 状态 |
|---|---|---|
| 14B v1-v3 | 实验 | 弃用 |
| 14B v4 | 实验 | 弃用 |
| 14B v5 | 5.8G LoRA | 实验 |
| 14B v6 (本仓) | 550MB LoRA | 生产 |
| 14B v6 merged | 28G | 生产 |
| 14B v6 q4 | 8.4G | 生产 |