StyQA is a unified quasi-agent framework for versatile style transfer tasks (pixel-level, semantic-level and continuous style transfer, etc.). This agent is attempted to be prompted with style transfer pipeline and normal style transfer instructions.
Typically, StyQA will conduct style-analysis $\rightarrow$ style-transfer $\rightarrow$ style-criteria pipeline, and iteratively refines the outputs based on the maximum refinement times. For convenient usage, we provide single stage calling, StyQA can run a single stage or combination of stages.
🚀 Quick Start
Environment
StyQA relies on uv and we provided pyproject.toml in this repo. Note that the embedded base model relies on the newest diffusers which should be installed with:
Computational Resource: Make sure your device satisfies the computation requirements:
The model with bfloat16 will comsume about 52G. If LoRA is used, each LoRA module consumes about 1G. So for better inference with at least $1024\times 1024$ pictures, $\geq 80$G is required.
Demos
main.py provides demos for continuous style transfer:
python
1@hydra.main(version_base="v1.2", config_path="configs", config_name="agent")2defmain(cfgs: OmegaConf):3 prompt ="Convert the pixel colors of Picture 1 into the pixel colors of Picture 2 with strength 0.75 and then transfer the semantic style into Picture 3."4 cnt_image_path ="demos/content1.jpg"5 ref_image_paths =["demos/pixel.jpg","demos/semantic1.jpg"]67 agent: StyQA = instantiate(cfgs.agent)8 user_input = UserInput(9 prompt=prompt,10 cnt_image_path=cnt_image_path,11 ref_image_paths=ref_image_paths,12)13 agent.run_pipeline(user_input)
Simply run
python main.py
and you will get the outputs like:
output-1
The agent configurations (seed, num_inference_steps, etc.) can be modified in configs/agent.yaml, or you can modify them using bash command thanks to hydra.
Note that the prompt can points a specific value and style transfer task types explicitly (it is what we recommanded) or with an implicit representation such as
"Using the colors and textures from Picture 2 ... moderate strength ... as if they are in same style category ..."
🖼️ Visualization
We provide more visualization performances.
Pixel-level style transfer
Similar to arbitrary image style transfer, Pixel-level style transfer aims to utilize the color and textures features from low level pixels.
pixel-level
Semantic-level style transfer
Artistic styles can be devided into different categories, semantic-level style transfer aims to re-generate content image using the same style category, which we call style-centric semantic features.
semantic-level
Continuous style transfer
Based on different pipeline commands, StyQA will parse them into workflow and conducts style transfer one-by-one, here we provide somes demos.
