LoRA adapter for Qwen/Qwen3.6-27B trained for Chinese sensitive-topic single-choice MCQ. Public LB: 0.77021 (NYCU AI/DL 2026 HW1, kaggle "1-llm-1-sft-on-answer-only-data").

Pipeline

Reasoning generation — base Qwen3.6-27B generates 1-2 sentence Chinese rationale per training Q
Pro-CCP filter — drop 7.3% of generated rationales that contain pro-CCP framing (keyword score)
SFT — QLoRA r=64 / α=128 on (Q + 4 options → "答案：X\n理由：[reasoning]"), lr=2e-5 cosine, 2 epochs (epoch 1 best)
DPO — 4 hard negative types (refusal / wrong-letter / vague / official-spin), lr=5e-6, β=0.2, 1 epoch
Inference — RAG retrieve top-10 (BGE-M3) → rerank top-3 (BGE-reranker-v2-m3) → vote=5 + TTP perm=4

Files

adapter_model.safetensors — final DPO LoRA weights (1.2 GB, r=64)
adapter_config.json — PEFT config
chat_template.jinja + tokenizer files — for inference

Usage

python

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                        bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.6-27B", quantization_config=bnb, device_map="auto",
    trust_remote_code=True, attn_implementation="eager",
)
model = PeftModel.from_pretrained(model, "jimhuang0731/qwen36-27b-v7-dpo-china-mcq")

Full pipeline code: https://github.com/Deep-Learning-NYCU/kaggle-1-llm-1-sft-on-answer-only-data-jimhuang0731

Training hyperparams

Stage	Setting	Value
SFT	LoRA r / α / dropout	64 / 128 / 0.05
SFT	target_modules	q,k,v,o + gate,up,down (MLP)
SFT	lr / scheduler	2e-5 / cosine, warmup 0.05
SFT	epochs / batch / accum	2 (ep1 best) / 1 / 8
DPO	lr / β	5e-6 / 0.2
DPO	epochs	1
Infer	vote × TTP perm	5 × 4 = 20 samples/Q
Infer	RAG retrieve / rerank	top-10 → top-3
Infer	temperature / top_p	0.6 / 0.95