This model, fine-tuned with LoRA to specialize in art criticism, is based on PaliGemma. It is trained on a diverse dataset that includes a wide range of artworks, from traditional paintings to contemporary art, focusing on 11 detailed aesthetic elements such as form, composition, and symbolism. The combination of PaliGemma's image and language processing capabilities with the LoRA fine-tuning technique provides users with insights into understanding the visual features and artistic intentions of a work from multiple perspectives.
🔍 Intended Uses & Limitations
GemmArte is designed for:
Analyzing artworks from various perspectives (formal, historical, emotional, technical, cultural, comparative)
Providing detailed interpretations of visual elements in paintings
Assisting in art education, appreciation, and research
Limitations:
The model's knowledge is limited to its training data and the base PaliGemma model
It may not accurately analyze very recent artworks or obscure art styles
Performance may vary depending on the quality and clarity of the input image
📊 Training dataset
The model training was conducted using the HuggingFace 🔗 Painting-Form dataset. This dataset consists of images of art pieces along with 215 questions and answers pertaining to each image. By leveraging this rich dataset, the model is able to provide detailed insights and critiques, enhancing its ability to understand and interpret various elements of art.
Datset key features
High-quality images of paintings
Detailed descriptions and analyses of these paintings
Description of the image
question
Please construct a formal analysis paragraph of approximately 175 words that is coherent and focuses solely on visual characteristics.\n<image>
answer
The painting depicts a young shepherdess, her gaze directed towards the viewer. Her figure is rendered with meticulous detail, her skin glowing with a soft, porcelain-like quality. The folds of her dress are depicted with a delicate touch, creating a sense of movement and fluidity...
🔄 Training Dataset example
The Painting-Form dataset was preprocessed by organizing the 215 questions into 11 categories. Each category is represented by a new, representative question (instruction), creating a refined training dataset. This approach enhances the model's ability to provide structured critiques, focusing on key aesthetic and interpretative elements of art.
1 prompt = "Provide a step-by-step formal analysis of this image, covering:1) Composition,2) Color,3) Line,4) Texture,5) Space,and6) Form.”
2'''
3 The composition of this artwork is striking, with a strong emphasis on the vertical and horizontal lines. The starry night sky dominates the canvas, filling the majority of the space with swirling patterns of light and dark. The use of bold, contrasting colors further enhances the dramatic effect of the scene, with the bright yellow stars against the deep blue sky creating a sense of tension and energy. The forms are simplified and stylized, with the only recognizable features being the church steeple and the distant village. The overall effect is one of a dynamic and expressive composition that captures the viewer's attention and draws them into the world of the painting.
4 '''56 prompt = "Please construct a formal analysis paragraph that is coherent and focuses solely on visual characteristic.”
7'''
8 The painting under consideration is a masterful example of the artist's use of color and brushwork to convey a sense of emotion and atmosphere. The dominant hues of blue and green create a striking contrast against the dark, swirling sky, which is rendered in a variety of shades of blue and white. The artist's use of thick, impasto brushstrokes adds texture and depth to the painting, particularly in the depiction of the swirling clouds and the distant mountains. The artist's attention to detail is evident in the intricate rendering of the stars and the church steeple, which are depicted with a combination of delicate, almost ethereal brushstrokes and bold, expressive strokes. The overall effect is a visually striking and emotionally charged work of art that captures the viewer's attention and invites them to contemplate the scene before them.
9 '''
The model will generate an analysis based on the specified prompt and the provided image. By using these structured prompts, you can guide the model to provide more detailed and comprehensive analyses of artworks.
](https://postimg.cc/8fh6G6dX)
](https://postimg.cc/8fh6G6dX)