# Download
huggingface-cli download yamura4/bbot bbot-qwen3.6-27b-Q4_K_M.gguf --local-dir .

# Serve
llama-server -m bbot-qwen3.6-27b-Q4_K_M.gguf --host 0.0.0.0 -c 32768 -ngl 100 --port 8080

LoRA adapter (requires base model)

Merge with base model using gguf-my-lora: https://huggingface.co/spaces/ggml-org/gguf-my-lora

Base model for GGUF: bartowski/Qwen_Qwen3.5-27B-GGUF

Or load directly with PEFT:

python

from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="yamura4/bbot",
    max_seq_length=2048,
)

Training details

Rank: 16, Alpha: 16
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
3 epochs, 500 samples
Trained with Unsloth + QLoRA on bbot security dataset