You can deploy this model and start earning money today!

Showmick119/
codellama-7b-fastapi-finetuned-20250713

Name: codellama-7b-fastapi-finetuned-20250713
Availability: InStock

codellama/CodeLlama-7b-Instruct-hf

llama2

Views

No views yet

CodeLlama-7B Fine-tuned for FastAPI Code Generation

This model is a fine-tuned version of codellama/CodeLlama-7b-Instruct-hf specialized for FastAPI code generation. It was trained using QLoRA (4-bit quantization + LoRA adapters) on 570 carefully curated FastAPI examples from real-world GitHub repositories.

Intended Use Case

Generate production-ready FastAPI code including endpoints, authentication, database integration, and error handling.

Intended Users

Python developers building FastAPI applications
Students learning FastAPI best practices
Teams needing rapid API prototyping

Training Details

Dataset

Size: 570 examples (augmented from 331 original patterns)
Source: Real-world FastAPI repositories from GitHub
Categories: Authentication (215), Database (94), Endpoints (74), Models (29), Validation (4)
Quality: Enhanced with contextual imports and complete code structure

LoRA Configuration

LoRA Rank (r): 32
LoRA Alpha: 64
LoRA Dropout: 0.1
Target Modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]
Quantization: 4-bit with bitsandbytes (nf4, double quantization)
Base Model: codellama/CodeLlama-7b-Instruct-hf

Training Hyperparameters

The following hyperparameters were used during training:

num_epochs: 3
train_batch_size: 2
gradient_accumulation_steps: 8
total_train_batch_size: 16
optimizer: paged_adamw_32bit with betas = (0.900, 0.999) and epsilon = 1e-08
learning_rate: 0.0001
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
seed: 42

Training Results

Step	Epoch	Training Loss	Validation Loss
10	0.3113	5.4923	4.4958
20	0.6226	0.4476	0.3011
30	0.9339	0.2841	0.2455
40	1.2179	0.2703	0.2151
50	1.5292	0.2241	0.1897
60	1.8405	0.1913	0.1625
70	2.1245	0.2357	0.1485
80	2.4358	0.1515	0.1357
90	2.7471	0.1373	0.1294

Initial Loss: 5.4923
Final Loss: 0.1294
Loss Reduction: 94%

Performance

Metric	Base Model	Fine-tuned	Improvement
FastAPI Code Quality	75.1/100	85.0/100	+9.9 points
Code Completeness	59.9/100	75.8/100	+15.9 points

Key Improvements:

Proper FastAPI imports and structure
Professional error handling with HTTP status codes
Database integration patterns (SQLAlchemy, MongoDB)
Authentication and validation logic
Production-ready code patterns

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "Showmick119/codellama-7b-fastapi-finetuned-20250713"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto",
    load_in_4bit=True
)

# Generate FastAPI code
prompt = "[INST] Create a FastAPI POST endpoint for user registration with email validation [/INST]"
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.1,
        do_sample=True,
        pad_token_id=tokenizer.pad_token_id
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response[len(prompt):].strip())

Limitations

Domain: Specialized for FastAPI; may not perform well on other frameworks
Context: Limited to 512 tokens; may truncate longer code examples
Dependencies: Requires GPU for optimal inference speed
Training Data Quality: GitHub-mined code was often messy and incomplete, requiring extensive preprocessing and enhancement to create complete, runnable training examples

Ethical Considerations

Code Quality: Generated code should be reviewed before production use
Security: Always validate generated authentication and security patterns
Licensing: Respects original training data licenses and CodeLlama terms

Citation

bibtex

@misc{codellama-fastapi-2025,
  title={CodeLlama-7B Fine-tuned for FastAPI Code Generation},
  author={Showmick119},
  year={2025},
  url={https://huggingface.co/Showmick119/codellama-7b-fastapi-finetuned-20250713}
}

Framework Versions

PEFT 0.16.0
Transformers 4.53.1
PyTorch 2.6.0+cu124
Datasets 4.0.0
Tokenizers 0.21.2
Accelerate 0.36.0
Bitsandbytes 0.45.0
TRL 0.12.0
HuggingFace Hub 0.26.0
NumPy 2.0.0
Python 3.10+

Additional Resources

Training Code: GitHub Repository
Training Notebook: Google Colab
Base Model: CodeLlama-7b-Instruct-hf