This model is a fine-tuned version of Meta-Llama-3.1-8B, optimized using Unsloth and QLoRA for superior instruction-following performance. It was trained on an NVIDIA L4 GPU for 2 epochs using the Alpaca-cleaned dataset.

📊 Training Performance (Technical Metrics)

The following metrics were captured at the end of the training session:

Final Training Loss: 0.9742
Training Runtime: 933.21 seconds (~15.5 minutes)
Training Throughput: 3.858 samples/second
Total Steps: 450
Epochs: 2.0
Hardware: NVIDIA L4 (24GB VRAM)

🏗️ Repository Structure

This repository is organized into three distinct deployment formats:

/merged_16bit: Full 16-bit Safetensors. Ideal for AWS Bedrock Custom Model Import and Amazon SageMaker.
/gguf: 4-bit quantized format (Meta-Llama-3.1-8B.Q4_K_M.gguf). Optimized for Ollama, llama.cpp, and local VM deployment.
/adapters: Raw LoRA adapters. Best for version control or continued incremental fine-tuning.

⚙️ Training Hyperparameters

Parameter	Value
LoRA Rank (r)	16
LoRA Alpha	32
Learning Rate	2e-4
LR Scheduler	Cosine Decay
Optimizer	AdamW 8-bit
Weight Decay	0.01
Max Seq Length	2048

🚀 Usage Guide

1. Fast Inference with Unsloth

python

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "CalmThinker/llama-3.1-8b-alpaca-finetune",
    max_seq_length = 2048,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

alpaca_prompt = """Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{}

### Response:
{}"""

inputs = tokenizer([alpaca_prompt.format("Explain the concept of 'quantization error'.", "")], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.batch_decode(outputs))