Business Assistant (Fine-tuned Qwen2.5-1.5B-Instruct)
Model Description
This is a LoRA fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct, specialized to answer Business questions only. It was trained on a synthetic instruction dataset generated using the Groq API (Llama 3.3 70B as the data-generation model), and explicitly trained to politely decline questions outside the Business domain rather than answering them.
- Developed by: [Dhwanit Rawat / joyboy5656n]
- Model type: Causal decoder-only language model (LoRA fine-tune, merged into base weights)
- Language(s): English
- License: Apache 2.0
- Finetuned from model: Qwen/Qwen2.5-1.5B-Instruct
Uses
Direct Use
Answering Business-related questions: startup strategy, business models, marketing, sales strategy, management, entrepreneurship, HR, negotiation, pricing strategy, competitive analysis, operations, and branding.
Out-of-Scope Use
This model is intentionally trained to refuse questions unrelated to Business (e.g. cooking, medical advice, coding, general trivia). It should not be relied on for domains outside Business, and it is not a substitute for professional financial, legal, or business consulting advice.
How to Get Started with the Model
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3repo = "your-hf-username/business-assistant-llm"
4tokenizer = AutoTokenizer.from_pretrained(repo)
5model = AutoModelForCausalLM.from_pretrained(repo)
6
7messages = [
8 {"role": "system", "content": "You are a specialized Business assistant. You ONLY answer questions related to Business. If a question is not about Business, politely decline."},
9 {"role": "user", "content": "How do I validate a startup idea before building it?"},
10]
11inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
12output = model.generate(inputs, max_new_tokens=300)
13print(tokenizer.decode(output[0], skip_special_tokens=True))
Training Details
Training Data
A synthetic dataset of instruction-response pairs generated via the Groq API, covering common Business topics, plus a set of refusal examples (~15% of the dataset) pairing off-domain questions with polite declines. Generated under a fixed daily API call budget to stay within Groq's free-tier limits.
Training Procedure
- Method: LoRA (Low-Rank Adaptation) fine-tuning, adapters merged into the base model after training
- Base model: Qwen2.5-1.5B-Instruct, loaded in 4-bit (NF4) for training
- LoRA config: r=16, alpha=32, dropout=0.05, applied to attention and MLP projection layers
- Epochs: 3
- Effective batch size: 16 (batch size 2 × gradient accumulation 8)
- Learning rate: 2e-4
- Training regime: fp16 mixed precision
Speeds, Sizes, Times
- Trained on a single Kaggle T4 GPU
- Dataset size: ~450 examples (~380 domain + ~70 refusal examples)
Evaluation
Manual qualitative evaluation was performed by prompting the model with in-domain Business questions and out-of-domain questions (e.g. cooking, sports) to verify it answers the former and declines the latter. No formal benchmark scores are reported.
Bias, Risks, and Limitations
- Trained on a small (~450 example) synthetic dataset, so coverage of niche Business subtopics may be limited.
- Synthetic training data was generated by another LLM (Llama 3.3 70B via Groq), so any biases or inaccuracies in that model's outputs may be inherited here.
- Not intended for use in place of professional business, legal, or financial consulting.
Environmental Impact
- Hardware Type: NVIDIA T4 GPU (Kaggle free tier)
- Hours used: < 1 hour
- Cloud Provider: Kaggle
- Compute Region: Unknown
Technical Specifications
- Architecture: Qwen2 (Qwen2.5 reuses the Qwen2 architecture class), ~1.5B parameters
- Software: transformers, peft, bitsandbytes, datasets, accelerate
Model Card Authors
[Dhwanit Rawat / joyboy5656n]
Model Card Contact
