Views
No views yet
fredzzp/open-dcoder-0.5B using error-aware training with a mixture objective. For detailed information on the training methodology, please refer to our paper: Corrective Diffusion Language Models.pip install torch transformers1import torch
2from transformers import AutoTokenizer, AutoModelForCausalLM
3
4model_id = "Shuibai12138/CDLM-0.5B"
5device = "cuda" if torch.cuda.is_available() else "cpu"
6
7# Load tokenizer and model
8tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
9model = AutoModelForCausalLM.from_pretrained(
10 model_id,
11 torch_dtype=torch.bfloat16,
12 trust_remote_code=True
13).to(device)
14
15# Generate code
16prompt = "def fibonacci(n):"
17input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
18
19# Use diffusion generation
20outputs = model.diffusion_generate(
21 inputs=input_ids,
22 max_new_tokens=100,
23 steps=16,
24 temperature=0.8
25)
26
27prompt_len = input_ids.shape[1]
28generated_text = tokenizer.decode(outputs.sequences[0][prompt_len:], skip_special_tokens=True)
29
30print("Generated Code:")
31print(generated_text)diffusion_generate method, so trust_remote_code=True is required when loading the model.1@misc{zhang2025correctivediffusionlanguagemodels,
2 title={Corrective Diffusion Language Models},
3 author={Shuibai Zhang and Fred Zhangzhi Peng and Yiheng Zhang and Jin Pan and Grigorios G. Chrysos},
4 year={2025},
5 eprint={2512.15596},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG},
8 url={https://arxiv.org/abs/2512.15596},
9}