You can deploy this model and start earning money today!

ssurface/
qwen3-4b-grpo-nolength-l1

Name: qwen3-4b-grpo-nolength-l1
Availability: InStock

Qwen/Qwen3-4B-Instruct-2507

finetune

apache-2.0

text-generation-inference

endpoints_compatible

Views

No views yet

Qwen3-4B GRPO — Level 1 (Verbose)

Qwen3-4B-Instruct fine-tuned with SFT → GRPO + new reward for compressed chain-of-thought reasoning at Level 1 (Verbose).

Training Pipeline

Qwen/Qwen3-4B-Instruct-2507
  → SFT LoRA (ssurface/qwen3-4b-cot-compress-l1)
  → Merged
  → GRPO with new reward
  → Merged (this model)

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("ssurface/qwen3-4b-grpo-nolength-l1")
tokenizer = AutoTokenizer.from_pretrained("ssurface/qwen3-4b-grpo-nolength-l1")

messages = [{"role": "user", "content": "Solve this using Level 1 (Verbose).\nProblem: ..."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))