interpolators/Fable-Qwen3-4B-SFT-bf16 is a compact Qwen3 4B instruction model fine-tuned on Fable 5 style agentic SFT traces and exported as merged bf16 weights.

The goal is a small, easy-to-run Fable-flavored model for coding, reasoning, and agentic chat experiments.

Recipe

Base model: unsloth/Qwen3-4B
Dataset: lordx64/agentic-distill-fable-5-sft
Training method: QLoRA / 4-bit loading with Unsloth
LoRA rank: 32
LoRA alpha: 64
Max sequence length: 2048
Training steps: 700
Effective batch: 1 x 8
Optimizer: adamw_8bit
Learning rate: 0.0002
Scheduler: cosine
Export: merged 16-bit/bf16 safetensors
Hardware: NVIDIA L4 24GB on Modal

Dataset Notes

The training data uses ChatML-style text examples from lordx64/agentic-distill-fable-5-sft, a Fable 5 distillation-style SFT dataset. No additional private data was added.

Intended Use

This model is intended for research and experimentation with Fable-style agentic behavior, compact assistant models, and Qwen3-family downstream adaptation.

Prompting

Use the Qwen chat template through tokenizer.apply_chat_template where possible.

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "interpolators/Fable-Qwen3-4B-SFT-bf16"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

messages = [
    {"role": "system", "content": "You are a helpful, careful assistant."},
    {"role": "user", "content": "Write a concise plan for debugging a failing training run."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tok.decode(out[0], skip_special_tokens=True))

Limitations

This is a quick research fine-tune and has not yet gone through a full benchmark suite. Evaluate carefully before relying on it for production use.

bingleai/Fable-Qwen3-4B-SFT-bf16

Fable Qwen3 4B SFT BF16

Recipe

Dataset Notes

Intended Use

Prompting

Limitations

bingleai/
Fable-Qwen3-4B-SFT-bf16