GRAM-Qwen3-4B-RewardModel-GGUF

GRAM-Qwen3-4B-RewardModel is a generative reward model developed to address reward generalization for Large Language Models (LLMs), released by NiuTrans. Unlike traditional models that depend heavily on task-specific labeled data, this model leverages both labeled and unlabeled data—a novel approach that allows it to generalize better across various tasks. It introduces a generative reward model framework that pre-trains on large amounts of unlabeled data and is subsequently fine-tuned with supervised data. The methodology also employs label smoothing and a regularized ranking loss to further boost performance, effectively bridging the gap between generative and discriminative reward modeling techniques.

This model is built on the Qwen3-4B base and can be directly used or adapted for aligning LLMs without the need to train a reward model from scratch on extensive datasets. In evaluations on the JudgeBench benchmark—covering Chat, Code, Math, and Safety tasks—GRAM-Qwen3-4B-RewardModel achieves a competitive average score of 65.9, making it suitable for use as an open-source, plug-and-play reward model for a variety of LLM alignment scenarios. The repository provides usage instructions and demonstration code to facilitate immediate adoption for research and development purposes

Model Files

Quants Usage