Views
No views yet
unsloth/llama-2-7b-bnb-4bit| Source | Count |
|---|---|
| ChatDoctor (Reformatted & Structured) | 500 |
| Refusal (High-Risk Queries) | 20 |
| Clarification — Ask | 20 |
| Clarification — Answer | 20 |
| Escalation (Emergency Cases) | 20 |
| General Knowledge | 20 |
| Out of Scope | 20 |
| Identity / System Persona | 20 |
| No Source Found | 22 |
| Total | 662 |
| Situation | Emoji |
|---|---|
| Danger / Disclaimer | 💀 |
| Out of Scope (Light Tone) | 🌚 |
| Serious Cases (Refusal / Clarification / Identity) | None |
1from unsloth import FastLanguageModel
2
3# Load base model
4model, tokenizer = FastLanguageModel.from_pretrained(
5 "unsloth/llama-2-7b-bnb-4bit",
6 load_in_4bit=True,
7)
8
9# Load TruthTriage adapter
10model.load_adapter("NayanPal/truthtriage-llama2-7b")
11
12# Inference
13FastLanguageModel.for_inference(model)
14
15inputs = tokenizer("Can I take Ibuprofen with Warfarin?", return_tensors="pt").to("cuda")
16outputs = model.generate(**inputs, max_new_tokens=200)
17
18print(tokenizer.decode(outputs[0], skip_special_tokens=True))