Views
No views yet
Qwen/Qwen3-8B (no supervised warm-start), optimizing a fixed rubric-based reward (GRPO).train/RL_task2_30K.jsonl, from SlowGuess/abforge-data
(derived from CC-licensed research papers). Evaluation uses the held-out AblationBench split
(eval/ablationbench_200.jsonl) of the same dataset.SlowGuess/ABForge-Qwen3-8B-Task2SlowGuess/ABForge-Qwen3-8B-Task2-SFTSlowGuess/ABForge-Qwen3-8B-Task2-RL (this model)SlowGuess/Abforge_1 code:1git clone https://github.com/SlowGuess/Abforge_1 && cd Abforge_1
2huggingface-cli download SlowGuess/abforge-data --repo-type dataset --local-dir data
3
4export MODEL_PATH=SlowGuess/ABForge-Qwen3-8B-Task2-RL
5
6# 1. Generate predictions on AblationBench (writes scorer-ready JSONL)
7python run_inference_local.py --task 2 \
8 --input data/eval/ablationbench_200.jsonl \
9 --output preds.jsonl \
10 --model-path "$MODEL_PATH" --dtype bf16 --device-map auto \
11 --max-new-tokens 4096 --temperature 0.0 --stop-on '</Proposed_Plan>'
12
13# 2. Score against the fixed 10-item AblationBench rubric
14export JUDGE_API_BASE=... JUDGE_API_KEY=... JUDGE_MODEL=...
15scripts/evaluate_task2.sh preds.jsonlSlowGuess/abforge-dataSlowGuess/Abforge_11@misc{abforge,
2 title = {ABForge: A Post-Training Pipeline for Paper-Grounded Ablation Design},
3 author = {ABForge authors},
4 year = {2026},
5 howpublished = {\url{https://github.com/SlowGuess/Abforge_1}}
6}