Views
No views yet
distilled_guidance_layer.layers) in my original upload. There is no need to download again if you are not having any issues with the older version.pattern_dict as input, but the original example code only provides it for Flux, which meant I had to learn the notation myself and modify it to fit other models. At least Chroma is just a pruned version of Flux, so it was relatively simple to derive the correct pattern_dict this time. Do let me know if you run into any problems.pattern_dict I used for compression:1pattern_dict = {
2 r"distilled_guidance_layer\.layers\.\d+": (
3 "linear_1",
4 "linear_2"
5 ),
6 r"transformer_blocks\.\d+": (
7 "attn.to_q",
8 "attn.to_k",
9 "attn.to_v",
10 "attn.add_k_proj",
11 "attn.add_v_proj",
12 "attn.add_q_proj",
13 "attn.to_out.0",
14 "attn.to_add_out",
15 "ff.net.0.proj",
16 "ff.net.2",
17 "ff_context.net.0.proj",
18 "ff_context.net.2",
19 ),
20 r"single_transformer_blocks\.\d+": (
21 "proj_mlp",
22 "proj_out",
23 "attn.to_q",
24 "attn.to_k",
25 "attn.to_v",
26 ),
27}diffusers1pip install dfloat11[cuda12]
2# or if you have CUDA version 11:
3# pip install dfloat11[cuda11]1import torch
2from diffusers import ChromaPipeline, ChromaTransformer2DModel
3from dfloat11 import DFloat11Model
4from transformers.modeling_utils import no_init_weights
5with no_init_weights():
6 transformer = ChromaTransformer2DModel.from_config(
7 ChromaTransformer2DModel.load_config(
8 "lodestones/Chroma",
9 subfolder="transformer"
10 ),
11 torch_dtype=torch.bfloat16
12 ).to(torch.bfloat16)
13
14pipe = Cosmos2TextToImagePipeline.from_pretrained(
15 "lodestones/Chroma",
16 transformer=transformer,
17 torch_dtype=torch.bfloat16
18)
19DFloat11Model.from_pretrained("mingyi456/chroma-unlocked-v48-detail-calibrated-DF11", device='cpu', bfloat16_model=pipe.transformer)
20pipe.enable_model_cpu_offload()
21prompt = "A high-fashion close-up portrait of a blonde woman in clear sunglasses. The image uses a bold teal and red color split for dramatic lighting. The background is a simple teal-green. The photo is sharp and well-composed, and is designed for viewing with anaglyph 3D glasses for optimal effect. It looks professionally done."
22negative_prompt = "low quality, ugly, unfinished, out of focus, deformed, disfigure, blurry, smudged, restricted palette, flat colors"
23image = pipe(
24 prompt,
25 negative_prompt=negative_prompt,
26 generator=torch.Generator("cpu").manual_seed(0)
27).images[0]
28image.save("chroma-unlocked-v48-detail-calibrated.png")pattern_dict that is of a completely different form compared to the one used to compress the model. If you are interested in running this model in ComfyUI, please try to contact the developer to request support.