Repository for Align-Anything Chameleon 7B Base, a powerful model for text-image interleaved input and output. This model is based on the Chameleon model, and is trained on the Align-Anything framework to further unlock its capability of image generation.

To use this model, you can refer to the Align-Anything repository for more details, including the training, inference and evaluation:

Then follow the instructions in the README.md file to set up the environment and run the scripts.

Currently, the official Transformer repo does not support Chameleon model with image output (see this PR for more details), so we rely on a certain fork of the repo.

After installing Align-Anything and correctly set up the envrionment, you can install the forked stable version of the repo by running:

If you want to generate image (pure text generation can be directly done by Transformers), you can follow the instructions in the mmsg_chameleon repo to run the inference.

Then set up the envrionment using

After setting up the envrioment, set up the correct paths in scripts/interleaved_gen.sh and then run