Fine-tuned TinyLlama for Utility Compliance Report Generation (UCRG)
This repository contains the fine-tuned weights for a TinyLlama-1.1B model, specialized for generating natural-language compliance reports from structured JSON data.
This model is the AI core of the Utility Compliance Report Generator (UCRG), a full-stack proof-of-concept project demonstrating an end-to-end AI application workflow.
The UCRG project showcases the entire lifecycle of a modern AI application:
Problem Definition: Automating the tedious task of writing compliance summaries from raw incident data.
Data Curation: Creating a custom, high-quality dataset for a specialized task.
Model Fine-tuning: Taking a powerful base model and adapting it to understand the specific domain language and format.
API Development: Building a robust Flask backend to serve the model's predictions.
Frontend & Deployment: Creating an intuitive React UI and deploying the full system to the cloud.
This model was trained to translate JSON objects describing utility incidents (like power outages or gas leaks) into clear, concise, human-readable summaries suitable for internal reporting or regulatory review.
Fine-tuning Technique: Low-Rank Adaptation (LoRA) using the peft and trl libraries.
Training Data: A custom-generated dataset of 50+ examples mapping JSON incident data to natural-language reports. The data was structured in an instruction format to optimize for the instruct base model.
Prompt Format
The model was trained using a specific instruction-following prompt format. To get the best results, your input should follow this structure:
### Instruction:
Generate a compliance report summary from the following structured data.
### Input:
{ "incident_type": "Power Outage", "duration": "3 hours", "location": "Zone 4", "resolved": true }
### Response:
🛠️ How to Use This Model
This model is served via a public Flask API as part of the full application. The easiest way to use it is through the live demo.
For programmatic use, you can load the base model and apply this LoRA adapter:
python
1import torch
2from peft import PeftModel
3from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
45# Define model paths6base_model_name ="TinyLlama/TinyLlama-1.1B-Chat-v1.0"7adapter_path ="rifukawa/ucrg-tinyllama-finetuned"89# Load tokenizer and base model10tokenizer = AutoTokenizer.from_pretrained(base_model_name)11base_model = AutoModelForCausalLM.from_pretrained(12 base_model_name,13 torch_dtype=torch.float16,14 device_map="auto",15)1617# Load the LoRA adapter18model = PeftModel.from_pretrained(base_model, adapter_path)1920# Create a text-generation pipeline21generator = pipeline("text-generation", model=model, tokenizer=tokenizer)2223# Prepare the prompt24prompt ="""### Instruction:
25Generate a compliance report summary from the following structured data.
2627### Input:
28{ "incident_type": "Gas Leak", "duration": "45 minutes", "location": "Substation B", "resolved": true }
2930### Response:
31"""3233# Generate the report34result = generator(prompt, max_new_tokens=100, do_sample=False)35print(result[0]['generated_text'].split("### Response:")[1].strip())3637# Expected output:38# A gas leak occurred at Substation B. The event lasted for 45 minutes. The issue has been resolved.
Limitations and Considerations
This model is a proof-of-concept and was trained on a small, simulated dataset.
Factuality: The model may "hallucinate" or fail to accurately represent all fields from the input JSON, especially for complex or unseen data structures. For example, it may incorrectly state an issue is resolved.
Scalability: For a production environment, this model would require a much larger and more diverse training dataset, along with a rigorous validation framework to ensure factual accuracy.
Despite these limitations, it effectively demonstrates the viability of using fine-tuned smaller LLMs for specialized, high-value automation tasks.
