Views
No views yet
unsloth/qwen3-14b-unsloth-bnb-4bit using PEFT and LoRA techniques, with a focus on legal consultation tasks grounded in Indian law. Its purpose is to assist users in understanding their legal rights, processes, and next steps through natural and client-friendly communication.unsloth/qwen3-14b-unsloth-bnb-4bit1!pip install -U bitsandbytes
2!pip install accelerate
3
4from peft import PeftModel
5from transformers import Qwen3ForCausalLM, AutoTokenizer, BitsAndBytesConfig
6import torch
7
8# Load base model
9base_model = Qwen3ForCausalLM.from_pretrained("unsloth/qwen3-14b-unsloth-bnb-4bit")
10
11# Load LoRA adapter
12model = PeftModel.from_pretrained(base_model, "hug-baahubali/LawyerGPT")
13
14# Load tokenizer
15tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")
16
17# Sample query
18query = "My husband says he moved all his money to his mom's account. How can I make sure the court still counts that as marital property?"
19
20# System prompt template
21system_prompt = '''You are a legal AI assistant specializing in Indian law. Your objective is to generate a comprehensive, actionable answer to each user’s legal question, tailored to their unique context.
22
23Guidelines:
241. Base your answer entirely on the question provided and your expert legal knowledge. When needed, supplement your response with information from reputable Indian legal or government sources.
252. Ensure every response is thorough, practical, and well-organized—at least 10 detailed sentences or bullet points.
263. Adopt the voice of a trusted lawyer advising a client—clear, realistic, and conversational.
274. Avoid academic, formal, or exam-style language. Do not use legal jargon without immediately explaining it in everyday terms.
285. When extra background or legal details are needed for a complete answer, identify these missing pieces and explain how obtaining this information would affect your advice.
296. Always cite essential legal provisions or procedural steps from authoritative Indian sources when you reference them.
307. Maintain the highest standards of legal accuracy, ethics, and client-centered communication.
318. Your primary goal is to empower each user—clarify their legal options, realistic risks or barriers, and next steps under Indian law.
32
33Every answer should leave the user feeling informed, supported, and equipped to take action.
34'''
35
36# Apply chat format
37chat = [
38 {'role': "system", "content": system_prompt},
39 {'role': "user", "content": query}
40]
41text_input = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
42
43from transformers import TextStreamer
44
45streamer = TextStreamer(tokenizer, skip_prompt=True)
46
47# Generate
48with torch.no_grad():
49 inputs = tokenizer(text_input, return_tensors="pt").to('cuda')
50 generated_ids = model.generate(
51 **inputs,
52 streamer=streamer,
53 max_new_tokens=512
54 )
55
56You can also try it in Colab (https://colab.research.google.com/drive/1eF-Pvcw5eJ2UXDED7H5hWeCvOy1fDg7T?usp=sharing)
57
58
59
60Framework versions:
61
62transformers: ≥4.39
63
64peft: 0.15.2
65
66accelerate: latest
67
68bitsandbytes: latest