Views
No views yet
meta-llama/Llama-2-7b-hf
using PEFT..
├── adapter_config.json # PEFT / LoRA hyper-parameters
├── adapter_model.bin # Trained adapter weights
├── README.md # This file
└── examples/
└── chat/
├── zero_shot/
│ └── prompt.json # Zero-shot chat prompt template
└── few_shot/
└── prompt.json # Few-shot chat prompt template| Strategy | Path | Description |
|---|---|---|
| Zero-shot | examples/chat/zero_shot/prompt.json | Single-turn; no demonstrations — the model relies on its instruction-following capability. |
| Few-shot | examples/chat/few_shot/prompt.json | Prepends three (user, assistant) demonstration turns before the live query. |
1from peft import PeftModel, PeftConfig
2from transformers import AutoModelForCausalLM, AutoTokenizer
3import json, pathlib
4
5# Load adapter config and base model
6config = PeftConfig.from_pretrained("dongbobo/adapter-checkpoint")
7base = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path)
8model = PeftModel.from_pretrained(base, "dongbobo/adapter-checkpoint")
9tok = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
10
11# Load a prompt template
12template = json.loads(
13 pathlib.Path("examples/chat/zero_shot/prompt.json").read_text()
14)
15
16# Build prompt
17user_msg = "Explain the concept of attention in transformers."
18prompt = (
19 f"<s>[INST] <<SYS>>\n{template['template']['system']}\n<</SYS>>\n\n"
20 f"{user_msg} [/INST]"
21)
22
23inputs = tok(prompt, return_tensors="pt")
24outputs = model.generate(**inputs, max_new_tokens=256)
25print(tok.decode(outputs[0], skip_special_tokens=True))| Parameter | Value |
|---|---|
| PEFT type | LORA |
| Task type | CAUSAL_LM |
Rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, v_proj |
| Bias | none |
meta-llama/Llama-2-7b-hf) is subject to its own
Llama 2 Community License.