smthem/

Kwai-Kolors-CoTyle-gguf

Innovative visual stylization is a cornerstone of artistic creation, yet generating novel and consistent visual styles remains a significant challenge. Existing generative approaches typically rely on lengthy textual prompts, reference images, or parameter-efficient fine-tuning to guide style-aware image synthesis, but often struggle with style consistency, limited novelty, and complex style representations. In this paper, we affirm that a style is worth one numerical code by introducing the novel task, code-to-style image generation, which produces images with novel, consistent visual styles conditioned solely on a numerical style code. To date, this field has only been primarily explored by the industry (e.g., Midjourney), with no open-source research from the academic community. To fill this gap, we propose CoTyle, the first open-source method for this task. Specifically, we first train a discrete style codebook from a collection of images to extract style embeddings. These embeddings are used to condition a text-to-image diffusion model (T2I-DM) for style-consistent generation. Subsequently, we train an autoregressive transformer on the quantized style codes to model their distribution, allowing the synthesis of novel style codes. During inference, a numerical code maps to a unique style sequence, which guides the diffusion process to produce images in the corresponding style. Unlike existing methods, our approach offers unparalleled simplicity and diversity, unlocking a vast space of reproducible styles from minimal input. Extensive experiments validate that CoTyle effectively turns a numerical code into a style controller, demonstrating a style is worth one code.

Run the following command to install the requirements.

Please download the checkpoints and put them to the ./pretrained_models directory. You can download them from Hugging Face.

For a quick walkthrough of the inference pipeline, we recommend generating a single image (see Single-Sample Generation). To intuitively experience the powerful capabilities of CoTyle, we recommend generating a batch of images (see Batch-Sample Generation), which by default produces 42 images (7 style codes × 6 prompts).

Run the following command to generate a batch of images. By default, 7 rows and 6 columns of images will be generated, where all images in each row are produced using the same style code, and all images in each column are generated using the same prompt. You can adjust the --style_code and the content in ./test_prompts.txt to obtain the desired outputs.

This process may take considerable time. Therefore, we provide an accelerated version based on piFlow, which requires only 4 denoising steps; however, this approach produces lower image quality. Enable --accelerate to activate piFlow.

If time permits, we strongly recommend executing the command below.

After successful execution, you will obtain the following results:

Execute the following code for single-sample inference. You can generate desired results by adjusting the --style_code and --prompt.

Similarly, you can enable the --accelerate to speed up.

We provide Gradio apps for interactivate inference with the CoTyle.

Official apps are available on HuggingFace Spaces.

If you want to run it locally, please execute:

Note: The Gradio apps use an accelerated version, which may result in a slight reduction in image generation quality.

Tips:

If CoTyle is helpful, please help to ⭐ the repo.

If you find this project useful for your research, please consider citing our paper:

This code builds on diffusers, Qwen-Image, piFlow and UniTok. Thanks for open-sourcing!