You can deploy this model and start earning money today!

ssurface/
qwen3-4b-gdpo-length-sft-l3

Name: qwen3-4b-gdpo-length-sft-l3
Availability: InStock

Qwen/Qwen3-4B-Instruct-2507

finetune

apache-2.0

text-generation-inference

endpoints_compatible

Views

No views yet

Qwen3-4B GRPO — Level 3 (Symbolic)

Qwen3-4B-Instruct fine-tuned with SFT → GRPO + new reward for compressed chain-of-thought reasoning at Level 3 (Symbolic).

Training Pipeline

Qwen/Qwen3-4B-Instruct-2507
  → SFT LoRA (ssurface/qwen3-4b-cot-compress-l3)
  → Merged
  → GRPO with new reward
  → Merged (this model)

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("ssurface/qwen3-4b-gdpo-length-sft-l3")
tokenizer = AutoTokenizer.from_pretrained("ssurface/qwen3-4b-gdpo-length-sft-l3")

messages = [{"role": "user", "content": "Solve this using Level 3 (Symbolic).\nProblem: ..."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))