Views
No views yet
negative / neutral / positive| Dataset | Base Accuracy | FT Accuracy | Base Macro-F1 | FT Macro-F1 |
|---|---|---|---|---|
| FPB in-domain | 0.8908 | 0.9748 | 0.8765 | 0.9725 |
| FiQA-SA OOD | 0.8120 | 0.9402 | 0.6705 | 0.8335 |
Baseline = zero-shot Meta-Llama-3.1-8B-Instruct with the same prompt template.
1from peft import PeftModel
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4base = AutoModelForCausalLM.from_pretrained(
5 "meta-llama/Meta-Llama-3.1-8B-Instruct",
6 load_in_4bit=True,
7 device_map="auto",
8)
9model = PeftModel.from_pretrained(base, "jhon53/Llama3_1_8B_Finance_QLoRA")
10tokenizer = AutoTokenizer.from_pretrained("jhon53/Llama3_1_8B_Finance_QLoRA")| Param | Value |
|---|---|
| Base model | meta-llama/Meta-Llama-3.1-8B-Instruct |
| Method | QLoRA (4-bit NF4 + LoRA bf16) |
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Target modules | q, k, v, o, gate, up, down projections |
| Training data | FinGPT/fingpt-sentiment-train (~76k) |
| Optimizer | AdamW 8-bit |
| Learning rate | 2e-4 (cosine schedule) |
| Epochs | 3 (early stopping, patience=3) |
| Loss | Completion-only cross-entropy |
llama-cli -hf jhon53/Llama3_1_8B_Finance_QLoRA-GGUF --jinjallama-mtmd-cli -hf jhon53/Llama3_1_8B_Finance_QLoRA-GGUF --jinjaMeta-Llama-3.1-8B-Instruct.Q4_K_M.gguf