Model Card
This model is a fine-tuned version of unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit.
The fine-tuning objective is to reduce hallucinations when an LLM is asked to map free-form construction job descriptions to the most appropriate equipment/work-type category.
Model Details
- Base model:
unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit
- Fine-tuning method: LoRA + SFT (PEFT / TRL)
- Task type: Domain classification-style text generation
- Primary domain: Construction permits / job description interpretation
Intended Use
- Convert user-provided job descriptions into grounded, domain-relevant categories.
- Improve consistency and reduce fabricated outputs versus the base model.
- Support permit-related assistant workflows where structured label output is needed.
Out-of-Scope Use
- Legal, compliance, or safety decisions without human review.
- Use outside construction/permit-like text domains without additional validation.
- High-stakes automation where incorrect classifications can cause harm.
Training Data
Fine-tuned on a sampled subset of NYC DOB permit records focused on:
Job DescriptionWork Type
Placeholder links to source datasets:
- Primary dataset (placeholder):
https://data.cityofnewyork.us/Housing-Development/DOB-NOW-Build-Approved-Permits/rbx6-tga4/about_data
Evaluation
The current notebook workflow tracks:
- Accuracy
- ROUGE-1
- ROUGE-2
- ROUGE-L
Please replace with run-specific values when publishing:
- Accuracy:
81.94%
- ROUGE-1:
82.03%
- ROUGE-2:
53.57%
- ROUGE-L:
82.03%
Limitations
- Can still hallucinate or over-generalize on unseen phrasing.
- Sensitive to class imbalance and noisy descriptions.
- Should be monitored for false positives across similar equipment/work types.
Get Started
Use the fine-tuned adapter/checkpoint from your training outputs and run inference with your existing prompt template for job-description classification.
Framework Versions
