Views
No views yet
1from unsloth import FastLanguageModel
2from peft import PeftModel
3
4# Load base model
5model, tokenizer = FastLanguageModel.from_pretrained(
6 "meta-llama/Meta-Llama-3.1-8B-Instruct",
7 load_in_4bit=True,
8)
9
10# Load adapter
11model = PeftModel.from_pretrained(model, "ekomobng/voira-essay-lora-v1")
12FastLanguageModel.for_inference(model)
13
14# Generate
15outline = """--- OUTLINE ---
16[your outline here]
17--- END ---
18
19Now write the essay based on this outline:"""
20
21messages = [
22 {"role": "system", "content": "You are an expert essay writer. Given a detailed outline, write the essay. Output ONLY the essay text, nothing else."},
23 {"role": "user", "content": outline},
24]
25
26input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
27inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
28
29outputs = model.generate(
30 **inputs,
31 max_new_tokens=2048,
32 temperature=0.7,
33 top_p=0.9,
34 repetition_penalty=1.1,
35 do_sample=True,
36)
37
38generated = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
39print(generated)--- OUTLINE --- and --- END --- markers, followed by the cue Now write the essay based on this outline:. The outline schema includes:[Prompt:] — the essay question or topic[Word Limit:] — target length[Overall Emotional Arc:] — macro emotional trajectory[Overall Tone & Voice:] — the writer's baseline register[Introduction / Hook:] — opening strategy with MOOD and STRESS notes[Body:] — numbered sections with content plans, key phrases, voice/pacing notes, MOOD, TONAL SHIFT, and STRESS[Conclusion / Summary:] — closing strategy with final emotional landing| Parameter | Value |
|---|---|
| Base model | meta-llama/Meta-Llama-3.1-8B-Instruct |
| Method | QLoRA (4-bit quantization + LoRA) |
| LoRA rank | 32 |
| LoRA alpha | 64 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Trainable params | 83.9M / 4.7B (1.78%) |
| Training examples | 91 |
| Validation examples | 15 |
| Epochs | 4 |
| Learning rate | 2e-4 |
| Scheduler | Cosine |
| Effective batch size | 8 (2 × 4 gradient accumulation) |
| Optimizer | AdamW 8-bit |
| Max sequence length | 8192 |
| Training loss (final) | 1.34 |
| Validation loss (final) | 1.70 |
| Hardware | NVIDIA A100 40GB |
| Training time | ~1.5 hours |
train_on_responses_only=True so the model is only scored on essay generation, not outline prediction.Base Llama 3.1 8B Instruct
└── voira-essay-lora-v1 (this adapter — general essay craft)
└── per-user LoRA (future — individual voice adaptation)