Rank-64 LoRA adapter for FLUX.1-dev, trained with critic-free GRPO using a frozen Q-Judger (Qwen-Image-Bench) as an online, multi-dimensional reward oracle.

⚠️ Preliminary research artifact. This adapter is the output of a 10-step exploratory run. It is released for reproducibility, not as a quality-improving adapter. No baseline comparison has been established — see Limitations.

Model Details

Base model: black-forest-labs/FLUX.1-dev
Method: GRPO (group-relative advantage, critic-free) with ODE→SDE sampling
Reward: frozen Q-Judger (Qwen-Image-Bench, Qwen3.6-27B), structured score vector over 5 pillars / 56 facets
Trainable params: 74.7M of 11.98B (0.62%)
Target modules: to_q, to_k, to_v, to_out.0 · rank 64, alpha 64
Developed by: Dinaiym Dubanaeva, Ala-Too International University, Bishkek, Kyrgyzstan

Training

Setting	Value
GRPO steps	10
Prompts / step	1
Samples / prompt (G)	3
Resolution	512×512, 6 denoising steps
Optimizer	AdamW, lr 1e-4
Clip ε / KL β	0.2 / 0.04
SDE noise σ	0.15
Prompts	Qwen-Image-Bench test split (1,000 prompts, `dims_en`)
Hardware	1× A100 80GB · 670 s/step · 1.86 h

Policy (12B) and judge (27B) do not co-reside in 80GB, so the phases are sequenced: each model is moved to CPU while the other occupies the GPU.

Results

Metric	Value
Q-Judger parse rate	1.000 (30/30 prompt–image pairs)
Gradient norm	332 – 764
Policy–reference KL	0 → 256
Mean reward	32.11 ± 11.62
OLS slope	+1.34/step (r² = 0.12, not significant)

The per-step reward is confounded by prompt difficulty (one prompt per step), so it must not be read as a learning curve.

Limitations

No baseline comparison yet. No claim of improved image quality is made.
Small scale. 10 steps, G=3, 512×512 — far below the regime where flow-matching GRPO is known to move benchmarks.
Evaluator circularity. The judge supplying the reward is also the judge reporting the score; KL reached 256 in 10 steps, so over-optimization cannot be ruled out without a held-out metric.

Usage

python

from diffusers import FluxPipeline
import torch

pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16
).to("cuda")
pipe.load_lora_weights("thedinaiym/judge-in-the-loop-flux-lora")

image = pipe("a robot walking in Tokyo", num_inference_steps=20).images[0]

Framework

PEFT 0.19.1 · diffusers 0.39.0 · transformers 5.13.0 · torch 2.12.0+cu130

thedinaiym/judge-in-the-loop-flux-lora

Judge-in-the-Loop — FLUX.1-dev LoRA (GRPO / Q-Judger reward)