This model is a fine-tuned version of CompVis/stable-diffusion-v1-4, trained on a small but rich dataset of 198 unique paintings by a single painter. It is optimized for generating text-to-image outputs with a distinctive hand-painted aesthetic.
This model card aims to document model details, usage recommendations, risks, and fine-tuning specifics in a transparent and reproducible manner.
Model Description
This model adapts Stable Diffusion v1.4 to replicate a specific human-created painting style. The training dataset includes 198 paintings in various themes and formats, designed to give the model a sense of color, brushwork, and composition typical to traditional art. It is suitable for generating stylized images with expressive, painterly textures. This model is for research purpose and discover how small dataset fine-tune can impact stable diffusion model behavior.
Below is a visual comparison between images generated by this fine-tuned model (xcheng20/stable-diffusion-painting-style-v1) and the base model (CompVis/stable-diffusion-v1-4) using the same prompts.
Prompt
Base Model Output
Fine-tuned Model Output
"Two very detailed owls with yellow eyes"
Base Model
Fine-tuned
"A phenix painted with white watercolor in the black background"
Base Model
Fine-tuned
"A modern city landscpae skyline in watercolor"
Base Model
Fine-tuned
Direct Use
This model is intended for artistic text-to-image generation. Prompt examples include:
"a peaceful cabin in the woods, painterly style"
"a surreal dreamscape in soft brushstrokes"
It is especially useful for artists, illustrators, and designers seeking an aesthetic similar to traditional hand-painted works.
Downstream Use
Artistic draft generation
Custom stylized prompt-to-image tools
Inspiration for illustration and concept art workflows
Out-of-Scope Use
Not suited for realistic portrait generation
Should not be used for any NSFW, violent, or biased content
Not recommended for medical, legal, or factual content generation
Bias, Risks, and Limitations
This model may not generalize well outside the stylistic patterns present in the dataset. It could reflect unintentional biases of the source style or create unrealistic outputs under complex prompts.
Recommendations
Avoid prompts involving sensitive content
Use with human review in artistic workflows
Not intended for factual accuracy or realism
How to Get Started with the Model
Option A: Download stable_diffusion_loader.py from the "Files and versions" tab, and run the code below:
python
1from stable_diffusion_loader import load_custom_pipeline, generate_image
23pipe = load_custom_pipeline("./fine-tuned-model")4image = generate_image(pipe,"Two very detailed owls with yellow eyes")5image.show()
