You can deploy this model and start earning money today!

pranav-pvnn/
codellama-7b-python-ai-assistant-full-gguf

Name: codellama-7b-python-ai-assistant-full-gguf
Availability: InStock

codellama/CodeLlama-7b-hf

quantized

llama2

endpoints_compatible

Views

No views yet

CodeLlama 7B Python AI Assistant (Merged GGUF)

This is a merged version of the QLoRA fine-tuned CodeLlama-7B model. The LoRA weights have been merged with the base model and converted to GGUF format for easy deployment.

Model Details

Base Model: CodeLlama-7b-hf
Original LoRA Adapter: pranav-pvnn/codellama-7b-python-ai-assistant
Fine-tuning Method: QLoRA (4-bit quantization with LoRA)
Format: GGUF (self-contained, no separate adapter needed)
Training Framework: Unsloth

Available Quantizations

codellama-7b-merged-f16.gguf - Full precision (FP16) - ~13 GB
codellama-7b-merged-Q4_K_M.gguf - 4-bit quantization (recommended) - ~4 GB
codellama-7b-merged-Q5_K_M.gguf - 5-bit quantization (higher quality) - ~5 GB
codellama-7b-merged-Q8_0.gguf - 8-bit quantization (highest quality) - ~7 GB

Usage

With llama.cpp:

./llama-cli -m codellama-7b-merged-Q4_K_M.gguf -p "### Instruction:\nWrite a Python function to calculate factorial.\n### Response:\n"

With Python (llama-cpp-python):

python

from llama_cpp import Llama

llm = Llama(model_path="codellama-7b-merged-Q4_K_M.gguf")
prompt = "### Instruction:\nWrite a Python function to calculate factorial.\n### Response:\n"
output = llm(prompt, max_tokens=256)
print(output['choices'][0]['text'])

With Ollama:

Create a Modelfile:

FROM ./codellama-7b-merged-Q4_K_M.gguf

Create the model:

bash

ollama create my-codellama -f Modelfile
ollama run my-codellama "Write a Python function to sort a list"

Training Details

Quantization: 4-bit QLoRA
LoRA Rank: 64
Learning Rate: 2e-4
Epochs: 4
Max Seq Length: 2048
Training Data: Custom Python programming examples (~2,000 examples)
GPU: NVIDIA Tesla T4

Prompt Format

### Instruction:
[Your instruction here]
### Response:

License

Same as base model (Llama 2 license)

Acknowledgements

Base Model: Meta's CodeLlama
Training Framework: Unsloth

pranav-pvnn/codellama-7b-python-ai-assistant-full-gguf