Model Card for Qwen HR Assistant (NovaTech Solutions)
Model Details
Model Description
This model is a fine-tuned version of Qwen2.5-0.5B-Instruct, adapted to act as an HR assistant chatbot for a fictional company, NovaTech Solutions. It answers employee questions about HR policies including leave (casual, sick, earned, unpaid, maternity, paternity), health/dental/vision insurance, wellness programs, performance reviews, travel & expense policies, IT/device usage, and onboarding.
The model was fine-tuned using QLoRA (4-bit quantized base model + LoRA adapters) on a custom instruction-response dataset, then the LoRA adapter was merged into the base model weights for standalone deployment.
Developed by: [adarsh-08]
Model type: Causal language model (decoder-only transformer), fine-tuned with LoRA/QLoRA
This model is intended for answering employee HR-policy questions in a Q&A/chatbot format, specifically for a single fictional company's internal policies (NovaTech Solutions). It is best used as a demo or internal FAQ assistant, not as a source of legally binding HR guidance.
Out-of-Scope Use
Not suitable for real HR/legal decision-making. Answers reflect a synthetic training dataset, not verified company policy or labor law, and may contain factual errors or invented details (see Limitations below).
Not designed for general-purpose chat — the model is narrowly fine-tuned on HR Q&A pairs and will underperform on unrelated tasks compared to the base instruct model.
Not intended for use with real employee personal data without independent verification of its outputs.
Bias, Risks, and Limitations
Hallucination risk on numbers. Being a small (0.5B parameter) model fine-tuned on a modest dataset, it can occasionally invent or blend numeric details (e.g., leave day counts, reimbursement amounts) that don't match the training data. Always verify specific figures against the actual source policy.
Narrow domain coverage. The model only knows what's in its ~500-example training set. Questions phrased very differently from training examples, or about topics not covered, may get low-quality, off-topic, or fabricated answers.
Synthetic company. All policy details (NovaTech Solutions) are fictional/for demonstration purposes.
Recommendations
Use this model for demonstration, prototyping, or as a starting point for a real internal HR assistant — not as an authoritative source. Any deployment handling real employee questions should include human review, a way to escalate to real HR staff, and further evaluation of factual accuracy before production use.
How to Get Started with the Model
python
1from transformers import AutoTokenizer, AutoModelForCausalLM
2import torch
34model_id ="your-username/qwen-hr-assistant"56tokenizer = AutoTokenizer.from_pretrained(model_id)7model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)89prompt ="Question: How many days of paid leave am I entitled to per year?\n\nAnswer:"10inputs = tokenizer(prompt, return_tensors="pt")1112output_ids = model.generate(13**inputs,14 max_new_tokens=200,15 do_sample=True,16 temperature=0.7,17 top_p=0.9,18 repetition_penalty=1.1,19 pad_token_id=tokenizer.pad_token_id,20 eos_token_id=tokenizer.eos_token_id,21)2223answer = tokenizer.decode(output_ids[0], skip_special_tokens=True)24print(answer)
Training Details
Training Data
A custom synthetic dataset of ~512 instruction-response pairs covering NovaTech Solutions' HR policies: leave types, insurance/benefits, wellness programs, performance reviews, travel & expenses, IT device policy, and onboarding. Multiple paraphrased questions map to the same canonical answer per topic, to encourage robustness to phrasing variation. Data was split 90/10 into train/validation sets.
Training Procedure
Fine-tuned using QLoRA: the base model was loaded in 4-bit (NF4) precision via bitsandbytes, and LoRA adapters were trained on top using the peft library. After training, the adapter was merged into the base model weights (merge_and_unload) for standalone deployment.
Preprocessing
Each example was formatted as:
Question: {instruction}
Answer: {response}
with an explicit EOS token appended, and a dedicated pad token (distinct from EOS) added to the tokenizer to avoid masking real end-of-sequence tokens during training.
Training regime: bf16 mixed precision (compute dtype), 4-bit NF4 quantized base weights during training
Epochs: 5
Per-device batch size: 4
Learning rate: 2e-4
Max sequence length: 256
Gradient checkpointing: enabled
Eval/save strategy: per epoch, best model (by validation loss) loaded at end
Speeds, Sizes, Times
Base model size: ~0.5B parameters
Hardware: Google Colab GPU runtime
Trained for 5 epochs on ~460 training examples / ~52 validation examples
Evaluation
Testing Data, Factors & Metrics
Testing Data
10% held-out split from the same synthetic dataset (random seed 42).
Metrics
Validation loss (cross-entropy) was used as the primary metric, with the best checkpoint (lowest validation loss) selected automatically at the end of training.
Results
Qualitative spot-checks after training showed accurate, on-topic, well-formatted answers for most HR policy questions. One coverage gap was identified (aggregate "total paid leave" questions defaulted to sick-leave-only answers) and fixed by adding targeted training examples in a follow-up training run.
Summary
The model performs well on in-distribution HR questions closely matching training phrasing and topics, with occasional numeric hallucination risk typical of small models trained on limited data. Recommended as a demo/prototype assistant rather than a production HR system without further validation.
Technical Specifications
Model Architecture and Objective
Decoder-only causal language model (Qwen2.5 architecture), fine-tuned via low-rank adaptation (LoRA) for the causal language modeling objective, then merged into the base weights.
