We introduce AceCoder-Qwen2.5-Coder-7B-Ins-V1.1, the updated model to the original AceCoder-Qwen2.5-Coder-7B-Base-Rule. We trained Qwen Coder 7B Base model with RL using AceCode-V1.1-69K dataset, and achieved impressive results, even surpassing Qwen Coder 2.5 7B Instruct. Proving the effectiveness of our dataset and RL for coding agents.

https://tiger-ai-lab.github.io/AceCoder/static/images/ac_overview.png

Note

This model official is trained on the TIGER-Lab/AceCode-V1.1-69K, using the binary pass rate (rule based reward) as the reward

To use the model, please refer to the codes in Qwen/Qwen2.5-7B-Instruct

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "TIGER-Lab/AceCoder-Qwen2.5-Coder-7B-Ins-V1.1"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

Performance

Model Name	LiveCodeBench-v4: (2023.5-2024.9)	HumanEval	HumanEval+	MBPP	MBPP+	BigCodeBench-Complete Full	BigCodeBench-Complete Hard	BigCodeBench-Instruct Full	BigCodeBench-Instruct Hard
GPT-4o (0806)	43.6	92.7	87.2	87.6	72.2	58.9	36.5	48.0	25.0
DeepCoder-14B-Preview	-	-	92.6	-	-	49.6	22.3	38.2	18.2
Qwen2.5-Coder-7B-Base (Backbone Model)	28.7	61.6	53.0	76.9	62.9	45.8	16.2	40.2	14.2
Qwen2.5-7B-Instruct	29.0	81.7	73.2	79.4	67.7	45.6	16.9	38.4	14.2
Qwen2.5-Coder-7B-Instruct	34.2	91.5	86.0	82.8	71.4	49.5	19.6	41.8	20.3
AceCoder-V1.1-7B	35.7	88.4	83.5	84.9	73.0	53.9	27.0	41.8	23.0

Citation

bibtex

@article{AceCoder,
    title={AceCoder: Acing Coder RL via Automated Test-Case Synthesis},
    author={Zeng, Huaye and Jiang, Dongfu and Wang, Haozhe and Nie, Ping and Chen, Xiaotong and Chen, Wenhu},
    journal={ArXiv},
    year={2025},
    volume={abs/2207.01780}
}