QLoRA 4-bit NF4 fine-tuning (r=16, α=32) on ~3.6k SFT examples. Part of writing-tools-nlp, which also implements speculative decoding, guided generation, and MLX on-device inference.

Evaluation (val split, n=400)

Metric	Base Qwen3-4B	Fine-tuned
ROUGE-1	0.3877	0.5359
ROUGE-2	0.1905	0.3788
ROUGE-L	0.3139	0.4838
BERTScore-F1	0.8756	0.9136

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3-4B-Instruct-2507"
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, "aditya-ice/writing-tools-qwen3")
tokenizer = AutoTokenizer.from_pretrained(base)

messages = [
    {"role": "system", "content": "You are a writing assistant. Summarise the following text in one concise paragraph."},
    {"role": "user", "content": "Your text here..."},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training

Base model: Qwen/Qwen3-4B-Instruct-2507
Method: QLoRA (peft + trl SFTTrainer)
Data: CNN/DailyMail, PAWS, Enron emails (~4k examples)
Hardware: Colab T4, 1 epoch

Adi-cookieNmilk/writing-tools-qwen3

writing-tools-qwen3

Approach

Evaluation (val split, n=400)

Usage

Training

Adi-cookieNmilk/
writing-tools-qwen3