You can deploy this model and start earning money today!

betancourtosmar/
fine-tuned-mistral-django-qa

Name: fine-tuned-mistral-django-qa
Availability: InStock

Views

No views yet

base_model: mistralai/Mistral-7B-Instruct-v0.2 library_name: peft tags:

llm
fine-tuned
mistral
django
python
docker
q-a
peft
lora datasets:
stackoverflow widget:
text: How do I set up a Django project with a PostgreSQL database in Docker? license: mit language:
en

Fine-tuned Mistral 7B for Programming Q&A (Django, Python, Docker)

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2, specifically adapted to answer programming-related questions, with a strong focus on Python, Django, and Docker technologies.

Model Details

Model Description

This model has been fine-tuned using LoRA (Low-Rank Adaptation), a Parameter-Efficient Fine-Tuning (PEFT) technique, combined with 4-bit quantization (bitsandbytes). This approach allowed for efficient adaptation of the large base model to a specialized domain without requiring extensive computational resources.

The primary goal of this fine-tuning was to enhance the model's ability to provide accurate, concise, and relevant responses to common development queries within the specified domains, making it a valuable tool for developers seeking quick answers and explanations.

Developed by: Osmar Betancourt
Model type: Instruction-tuned Large Language Model (LLM)
Language(s) (NLP): English
License: MIT License
Finetuned from model: mistralai/Mistral-7B-Instruct-v0.2

Model Sources

Repository: https://github.com/osmarbetancourt/osmar-generative-ai (This repository contains the fine-tuning scripts and data acquisition logic.)
Demo: An interactive demo is integrated into my main portfolio website.

Uses

Direct Use

This model is intended for direct use as a specialized AI assistant for programming questions, particularly in the areas of Python, Django, and Docker. Users can query it for:

Explanations of programming concepts.
Guidance on Django framework usage.
Troubleshooting Docker environments.
General Python development queries.
Code explanations and best practices.

Downstream Use

This model can be integrated into larger applications or ecosystems, such as:

Developer support tools or chatbots.
Code IDE extensions for quick answers.
Educational platforms for interactive learning.
Automated documentation generation (with further fine-tuning).

Out-of-Scope Use

This model is not intended for:

Generating harmful, unethical, or illegal content.
Providing medical, legal, or financial advice.
Performing tasks outside its fine-tuned domain (e.g., creative writing unrelated to programming, general knowledge outside IT).
Applications requiring absolute factual accuracy without human verification.
High-stakes decision-making.

Bias, Risks, and Limitations

Like all large language models, this fine-tuned model may exhibit:

Hallucinations: Generating factually incorrect or nonsensical information.
Outdated Information: Its knowledge is based on its training data, which has a cutoff date and may not reflect the absolute latest changes in frameworks or libraries.
Bias: Reflecting biases present in its pre-training and fine-tuning data.
Limited Scope: While specialized, it may not perform as well on highly niche or esoteric programming topics not well-represented in its training data.
Truncation: Responses are limited by max_new_tokens during inference, which may lead to incomplete answers for complex queries.

Recommendations

Users should be aware of these limitations and use the model's output as a helpful guide, always verifying critical information. Human oversight and review are recommended for any high-impact applications.

How to Get Started with the Model

Use the code below to load and get started with the model using the transformers library.

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

# Define the same quantization configuration used during fine-tuning
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

# Load the fine-tuned model and tokenizer
# Replace 'betancourtosmar/fine-tuned-mistral-django-qa' with the actual model ID if different
model_id = "betancourtosmar/fine-tuned-mistral-django-qa"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=quantization_config,
    device_map="auto",
    torch_dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16,
)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Ensure pad_token is set for generation
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right" # Important for batch inference

model.eval() # Set the model to evaluation mode

def generate_response(user_input: str, max_new_tokens: int = 200) -> str:
    messages = [{"role": "user", "content": user_input}]
    # Apply chat template for instruction-tuned models like Mistral
    formatted_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    
    inputs = tokenizer(formatted_prompt, return_tensors="pt").to(model.device)

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=0.7,
            top_p=0.9,
            num_return_sequences=1,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.pad_token_id # Ensure pad_token_id is passed
        )

    generated_text_with_prompt = tokenizer.decode(outputs[0], skip_special_tokens=False)
    
    # Extract only the assistant's response
    import re
    match = re.search(r'\[/INST\]\s*(.*)', generated_text_with_prompt, re.DOTALL)
    if match:
        generated_text = match.group(1).strip()
    else:
        generated_text = generated_text_with_prompt.strip()

    # Clean up any remaining special tokens if skip_special_tokens=False was used
    generated_text = generated_text.replace("<s>", "").replace("</s>", "").strip()
    
    return generated_text

# Example usage:
# response = generate_response("Explain how to set up a Django project with a PostgreSQL database.")
# print(response)

Training Details

Training Data

The model was fine-tuned on a custom dataset (combined_qa_dataset.jsonl) generated by programmatically acquiring data from the Stack Overflow API. This approach ensured that the data was relevant to programming questions and adhered to Stack Exchange's licensing terms (Creative Commons Attribution-ShareAlike 4.0 International - CC BY-SA 4.0).

Data Acquisition: The stack_exchange_api_acquisition.py script was used to fetch questions and their accepted answers based on specific tags (python, django, docker). No large data dumps were used; all data was fetched directly via the API.
Data Processing: HTML content from questions and answers was cleaned, and the data was formatted into instruction-response pairs ({"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}) suitable for instruction fine-tuning. The data is anonymized as provided by the Stack Overflow API.
Dataset Size: The combined dataset contains approximately 10,000 to 13,000 unique Q&A pairs (depending on the exact acquisition run and filtering).

Training Procedure

Preprocessing

The raw HTML content from Stack Overflow questions and answers was cleaned to remove extraneous tags, preserve code blocks (using Markdown triple backticks), and normalize whitespace. The cleaned text was then formatted into the messages structure required for instruction fine-tuning with the trl library.

Training Hyperparameters

Training regime: Mixed precision (bfloat16 compute dtype for 4-bit quantization)
Epochs: 1
Learning Rate: 2e-4
Optimizer: AdamW with 8-bit quantization (bitsandbytes)
LoRA Parameters:
- lora_alpha: 16
- lora_dropout: 0.1
- r: 8
Max Sequence Length: 512 tokens
Batch Size: 1 (gradient accumulation used to simulate larger batches)
Gradient Accumulation Steps: 4
Warmup Ratio: 0.03
Weight Decay: 0.001

Speeds, Sizes, Times

GPU Used: NVIDIA GeForce RTX 4070 Ti (12GB VRAM)
Training Time: Approximately 3-4 hours for 1 epoch on the specified hardware and dataset size.
Checkpoint Size: LoRA adapters are relatively small (e.g., a few hundred MBs), significantly less than the full model. The merged model (pushed to Hub) is larger but still efficient due to 4-bit quantization.

Evaluation

Testing Data, Factors & Metrics

Testing Data

Qualitative evaluation was performed using a set of Django, Python, and Docker-related prompts not explicitly present in the training data.

Factors

Relevance of response to the prompt.
Accuracy of technical information.
Conciseness and clarity of explanation.
Ability to generate code snippets where appropriate.

Metrics

No formal quantitative metrics (e.g., BLEU, ROUGE) were used for this initial fine-tuning. Evaluation was primarily qualitative and based on human judgment of response quality.

Results

Summary

The fine-tuned model demonstrates an improved ability to answer domain-specific questions compared to the base model, providing more focused and technically accurate responses related to Python, Django, and Docker. It can generate relevant code snippets and explain concepts clearly. However, like all LLMs, it is not immune to hallucinations and requires human verification for critical information.

Model Examination

No specific interpretability work has been performed on this model.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: NVIDIA GeForce RTX 4070 Ti
Hours used: ~4 hours (for 1 epoch)
Cloud Provider: Local machine (no cloud provider emissions for training)
Compute Region: N/A (local)
Carbon Emitted: (Calculation requires specific power consumption data, but estimated to be low given short training time and single GPU)

Technical Specifications

Model Architecture and Objective

The model utilizes the Mistral-7B-Instruct-v0.2 architecture. The fine-tuning objective was to minimize the language modeling loss on the instruction-response pairs, effectively teaching the model to follow programming-related instructions and generate informative answers.

Compute Infrastructure

Hardware

GPU: NVIDIA GeForce RTX 4070 Ti (12GB VRAM)
CPU: Intel Core i7 (or equivalent)
RAM: 32GB+

Software

Operating System: Windows 11 (with WSL2 for Docker/GPU passthrough)
Conda Environment: ai_dev_env
Python: 3.10
Frameworks/Libraries:
- torch==2.3.0+cu121
- transformers==4.36.2
- tokenizers==0.15.0
- peft==0.10.0
- accelerate==0.29.3
- datasets==2.20.0
- trl==0.8.6
- numpy==1.26.4
- fsspec==2024.5.0
- fastapi==0.111.0
- uvicorn==0.30.1
- gradio==4.37.1
- requests
- beautifulsoup4
- python-dotenv
- huggingface_hub==0.23.0

Citation

BibTeX:

bibtex

@misc{betancourt2025finetunedllm,
  author = {Betancourt, Osmar},
  title = {Fine-tuned Large Language Model for Programming Q\&A (Django, Python, Docker)},
  year = {2025},
  publisher = {Hugging Face},
  url = {[https://huggingface.co/betancourtosmar/fine-tuned-mistral-django-qa](https://huggingface.co/betancourtosmar/fine-tuned-mistral-django-qa)}
}

APA:

Betancourt, O. (2025). Fine-tuned Large Language Model for Programming Q&A (Django, Python, Docker). Hugging Face. Retrieved from https://huggingface.co/betancourtosmar/fine-tuned-mistral-django-qa

Glossary

LLM (Large Language Model): A type of artificial intelligence model designed to understand and generate human-like text.
PEFT (Parameter-Efficient Fine-Tuning): A family of techniques that enable efficient adaptation of pre-trained LLMs to downstream tasks without fine-tuning all of the model's parameters.
LoRA (Low-Rank Adaptation): A specific PEFT method that injects small, trainable rank-decomposition matrices into the existing layers of a pre-trained model.
Quantization: A technique to reduce the precision of the numbers used to represent a model's weights and activations, thereby reducing memory usage and speeding up inference.
Hallucination: The phenomenon where an LLM generates plausible-sounding but factually incorrect or nonsensical information.

More Information

For detailed scripts and further context, please refer to the project's GitHub repository: https://github.com/osmarbetancourt/osmar-generative-ai

Model Card Authors

Osmar Betancourt

Model Card Contact

oaba.dev@gmail.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.