Views
No views yet
1from peft import PeftModel
2from transformers import AutoModelForCausalLM, AutoTokenizer
3import torch
4
5# Load base model
6model = AutoModelForCausalLM.from_pretrained(
7 "mistralai/Mistral-7B-Instruct-v0.3",
8 torch_dtype=torch.bfloat16,
9 device_map="auto"
10)
11
12# Load LoRA adapter
13model = PeftModel.from_pretrained(model, "paraluxdotai/compliance-7b-lora")
14tokenizer = AutoTokenizer.from_pretrained("paraluxdotai/compliance-7b-lora")
15
16# Generate
17prompt = """<s>[INST] You are Paralux Compliance Assistant, a specialized AI for financial regulatory compliance.
18
19Convert this requirement to a compliance rule: All financial documents must be retained for 7 years [/INST]"""
20
21inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
22outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
23print(tokenizer.decode(outputs[0], skip_special_tokens=True))| Parameter | Value |
|---|---|
| Base model | mistralai/Mistral-7B-Instruct-v0.3 |
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Training examples | 14,918 |
| Evaluation examples | 1,658 |
| Epochs | 3 |
| Final eval loss | 0.084 |
| Learning rate | 1e-4 |
| Batch size | 16 (4 × 4 gradient accumulation) |
| Precision | bfloat16 |
1@misc{paralux-compliance-7b-lora,
2 author = {Paralux},
3 title = {Paralux-Compliance-7B-LoRA: Fine-tuned Mistral for Financial Compliance},
4 year = {2026},
5 publisher = {Hugging Face},
6 url = {https://huggingface.co/paraluxdotai/compliance-7b-lora}
7}