A domain-specific AI assistant fine-tuned on official AWS documentation using QLoRA (Quantized Low-Rank Adaptation). Answers questions about AWS services with grounded, accurate responses and refuses out-of-scope questions.

Model Details

Model Description

This model is a QLoRA fine-tune of Qwen2.5-7B-Instruct, trained on 407 instruction-following Q&A pairs generated from official AWS documentation. It is designed to answer questions about AWS cloud services accurately, with built-in domain refusal for non-AWS questions. The full system includes FAISS-based retrieval (RAG) and a FastAPI backend.

Developed by: Debarun Ghosh (Debarun12)
Model type: Causal Language Model (instruction-tuned, QLoRA adapter)
Language: English
License: Apache 2.0
Finetuned from: Qwen/Qwen2.5-7B-Instruct
PEFT Version: 0.13.0

Model Sources

Repository: https://github.com/Debarun12/aws-enterprise-ai-assistant
HuggingFace: https://huggingface.co/Debarun12/aws-enterprise-assistant-qwen2.5-7b-qlora

Uses

Direct Use

Load the adapter on top of Qwen2.5-7B-Instruct to answer AWS-related questions about EC2, S3, Lambda, IAM, RDS, VPC, CloudWatch, and DynamoDB.

Downstream Use

Plug into a RAG pipeline with a FAISS vector store built from AWS documentation chunks for grounded, source-cited answers. A FastAPI backend with streaming support is included in the project repository.

Out-of-Scope Use

This model is not intended for general-purpose conversation, code generation, or non-AWS topics. It will refuse questions outside the AWS domain by design.

How to Get Started with the Model

python

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_name = "Qwen/Qwen2.5-7B-Instruct"
adapter_name = "Debarun12/aws-enterprise-assistant-qwen2.5-7b-qlora"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(adapter_name, trust_remote_code=True)

# Load base model in 4-bit
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_name)
model.eval()

# Inference
messages = [
    {"role": "system", "content": "You are an expert AWS cloud assistant."},
    {"role": "user", "content": "What is the difference between IAM roles and IAM users?"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    outputs = model.generate(**inputs, max_new_tokens=300, temperature=0.3, do_sample=True)

answer = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(answer)

Training Details

Training Data

Source: Official AWS documentation (publicly available at docs.aws.amazon.com)
Services: EC2, S3, Lambda, IAM, RDS, VPC, CloudWatch, DynamoDB
Pages scraped: 53 documentation pages
Chunks: 211 text chunks (~460 words average)
Q&A pairs: 407 instruction-following examples
Generation model: Qwen2.5-7B via Ollama (local)
Question types: Factual (211), Procedural (196)

Training Procedure

Preprocessing

Raw HTML pages were scraped from AWS documentation, cleaned with BeautifulSoup, and chunked into 512-word overlapping segments. Each chunk was passed to a local Qwen2.5-7B model to generate factual and procedural Q&A pairs in Alpaca instruction format. Pairs were formatted using Qwen2.5's chat template via apply_chat_template.

Training Hyperparameters

Training regime: bf16 mixed precision
LoRA rank (r): 8
LoRA alpha: 16
LoRA dropout: 0.05
Target modules: q_proj, v_proj
Quantization: 4-bit NF4 with double quantization
Optimizer: paged_adamw_8bit
Learning rate: 2e-4
LR scheduler: cosine
Epochs: 3
Batch size: 1 (effective 16 with gradient accumulation)
Max sequence length: 512 tokens
Warmup ratio: 0.05

Speeds, Sizes, Times

Hardware: NVIDIA GeForce RTX 5050 (8.5GB VRAM)
Training time: ~17 minutes
Adapter size: ~9.8MB (adapter_model.safetensors)
Trainable parameters: 2,523,136 / 7,618,139,648 (0.03%)

Evaluation

Testing Data

9 held-out questions covering all 8 AWS services, plus 3 out-of-scope questions for refusal testing.

Metrics

Confidence scores are derived from cosine similarity between query embeddings and retrieved documentation chunks (FAISS IndexFlatIP with normalized vectors).

Results

Question Type	Avg Confidence	Refused Correctly
S3 (encryption, storage)	0.724	N/A
IAM (roles vs users)	0.686	N/A
Lambda (concurrency)	0.680	N/A
CloudWatch (monitoring)	0.783	N/A
DynamoDB (components)	0.694	N/A
DynamoDB (high availability)	0.739	N/A
DynamoDB (auto-scaling)	0.688	N/A
Out-of-scope (3 questions)	0.000	✅ 100%

Summary

The model achieves consistent confidence scores of 0.68–0.78 on AWS domain questions and correctly refuses 100% of out-of-scope questions. Answers are grounded in retrieved AWS documentation via FAISS RAG.

Bias, Risks, and Limitations

AWS documentation changes over time; the training data reflects a snapshot from June 2026.

Recommendations

Always verify critical AWS configuration decisions against the official AWS documentation. Use the confidence score as a signal — responses below 0.5 should be treated with caution.

Environmental Impact

Hardware: NVIDIA GeForce RTX 5050 (laptop GPU)
Hours used: 0.28 hours (~17 minutes)
Cloud Provider: None (local training)
Carbon Emitted: Negligible (local, short training run)

Technical Specifications

Model Architecture

Base: Qwen2.5-7B-Instruct (decoder-only transformer)
Adapter: LoRA on q_proj + v_proj attention layers
Inference: 4-bit NF4 quantized base + bf16 LoRA computation
RAG: FAISS IndexFlatIP with sentence-transformers/all-MiniLM-L6-v2 embeddings

Software

PyTorch 2.12.0 (CUDA 12.8)
Transformers 4.46.0
PEFT 0.13.0
TRL 0.9+
FAISS 1.14.3
Sentence-Transformers 5.5.1
FastAPI 0.112+

Model Card Authors

Debarun Das — GitHub | HuggingFace

Framework Versions

PEFT 0.13.0
Transformers 4.46.0