We build on the work of GameNGen and show that diffusion models can learn multiple games of diffrent complexities. In this repository, we have open sourced the code for the community to build upon our work.
The models which we have trained can successfully run the following games:
Chess
Snake and Food
Car Obstacle Avoid
Conway's Game of Life (64x64 Grid)
This repository contains dataset generation files, model training and finetuning code, inference and other utilities used for documentation. We have, also, included weights of the trainined models. Inference requires atleast 8GB of vRAM on windows computers. Inference code will run on all ARM Based MacBooks.
Generated Images:
Image 1
Image 2
Image 3
Why a Single Diffusion Model?
Shared Visual & Temporal Priors
Diffusion models excel at learning pixel‑level structures and motion dynamics. A single model can capture these common patterns.
Proven Video Diffusion Robustness
Surveys show diffusion models achieve high fidelity and temporal consistency in video tasks. This robustness underpins stable next‑frame generation across multiple game domains.
Foundation‑Model Simplicity
Treating next‑frame prediction as a “foundation” task aligns with best practices in multimodal AI: one core model serves many downstream applications via lightweight conditioning. This cuts engineering overhead and streamlines deployment.
Efficient Auto‑Regressive Rollouts
Research on game‑engine diffusion (e.g., DOOM) demonstrates stable, long‑duration rollouts (> 20 FPS) with a single model on modest hardware. Extending to four games leverages the same auto‑regressive denoising pipeline without per‑game retraining.
Insights into Universal Game Dynamics
By training on multiple games, the model reveals which visual and rule‑based features are truly shared versus domain‑specific. These insights can inform procedural generation, asset reuse, and hybrid‑genre design.
Training
This part will document in detail the setup used to create the pipeline, dataset, and models. Follow this documentation for a quick and easy setup!
Configure the number of episodes and number of transitions inside the python dataset files. The data will be stored in ~/dataset/ folder. dataset folder will be created if it doesn't exist.
[!WARNING]
Current configuration of number of episodes require ~700 GB of memory -- choose the episode number depending on your local machine's capacity.
Stable Diffusion 1.5v VAE is trained on real life images which have curves and irregular edges. This is not ideal for our games -- the games we choose have lots of straight lines and 90 degree edges.
By Finetuning the VAE decoder, we preserve the VAE encoder's downsampling ability and enchance the decoder's upsampling ability for our game domain.
We will repurpose the stable diffusion 1.5v Unet to predict next frame by sending a previous latent as input conditioned on the previous action. The code will create a new file and dump checkpoints into. Restarting training will also be done from the checkpoint. We trained the model for 10 epochs on four Nvidia H100 GPUs for 20 hours.
accelerate launch train.py
Inference
Put the ./training_diffusion/ into your Desktop or working directory
Run the above inference.ipynb in the working directory.
Usage
Usage
Game
Command
Description
Controls
Chess
<chess>
Displays the starting board
UCI moves (e.g. e2e4, a2a3)
Car Obstacle Avoid
<car>
Displays the starting frame
up, left, right
Snake and Food
<snake_n_food>
Displays the starting frame
up, down, left, right
Conway’s Game of Life
<game_of_life>
Displays the starting grid
A (advance one step)
Exit
<exit>
Exit any game
—
📖 Citation
If you use YAGANG in your research, please cite:
bibtex
1@software{yagang2025,
2 author = {Chinmay S, Rachit M, Sharaajan G, and Anok S},
3 title = {YAGANG: Yet Another Game NGen},
4 year = 2025,
5 publisher = {GitHub},
6 url = {https://github.com/GenAI-2025-Project/YAGANG-YetAnotherGAmeNgen-}
7}
