This repository contains a QLoRA adapter for Qwen/Qwen2.5-72B-Instruct, fine-tuned using the Glint-Research/Fable-5-traces dataset via Unsloth.

The goal of this project is to perform Behavioral Cloning, distilling the deep multi-turn planning, tool-use logic, and explicit Chain-of-Thought (CoT) behaviors from the Fable-5 agent model into the powerful 72B architecture of Qwen 2.5.

🚀 Model Capabilities

Explicit Reasoning: The model is trained to think before responding, encapsulating its step-by-step logic and planning inside <think>...</think> tags.
Agentic Mindset: Improved performance in multi-turn agentic workflows, complex problem decomposition, and tool-use simulation based on actual Fable-5 execution traces.
Retained Core Strength: Built on top of Qwen2.5-72B-Instruct, maintaining its elite coding, mathematical, and general instruction-following capabilities.

🛠️ Training Details

The model was trained using the following configurations and hyperparameters:

Framework: Unsloth (2x faster free finetuning)
Method: QLoRA (4-bit quantization with LoRA adapters)
Base Model: Qwen/Qwen2.5-72B-Instruct-bnb-4bit
Dataset: Glint-Research/Fable-5-traces (Merged CoT and agent traces)
Max Sequence Length: 4096 tokens
LoRA Configuration:
- Rank ($r$): 16
- Alpha ($\alpha$): 32
- Target Modules: All linear layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
Optimization:
- Optimizer: adamw_8bit
- Learning Rate: 2e-4
- Batch Size per device: 2
- Gradient Accumulation Steps: 4
- Precision: bfloat16 / float16 hybrid tracking

💻 How to Use

Loading the Adapter with Unsloth (Recommended)

python

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "armzanaja/qwen2.5-72b-fable5-lora",
    max_seq_length = 4096,
    load_in_4bit = True, # Set to False if you have enough VRAM for 16-bit
)
FastLanguageModel.for_inference(model) # Enable 2x faster inference

# Example Prompt
messages = [
    {"role": "user", "content": "Create a python script to analyze system logs and detect anomalies."}
]

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=2048, temperature=0.7)

print(tokenizer.decode(outputs[0], skip_special_tokens=False))

📦 How to Convert this LoRA to GGUF for Ollama

Since this repository contains only the lightweight LoRA adapter, you need to merge it with the base Qwen model before creating a GGUF file. You don't need an expensive GPU for this process—a standard machine with ~100GB of disk space and CPU setup can do it locally using llama.cpp:

Local Conversion via llama.cpp (CPU/Disk Only)

bash

# 1. Clone llama.cpp repository
git clone [https://github.com/ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp)
cd llama.cpp
pip install -r requirements.txt

# 2. Download the official conversion script tool
# 3. Run the conversion script by pointing to the Base Qwen model and adding your LoRA adapter path
python convert_hf_to_gguf.py --outfile qwen2.5-72b-fable5-Q4_K_m.gguf --outtype q4_k_m --lora-base Qwen/Qwen2.5-72B-Instruct armzanaja/qwen2.5-72b-fable5-lora