You can deploy this model and start earning money today!

Gavit0/
InfarctImage

Name: InfarctImage
Rating: 2.5 (6 reviews)

stabilityai/stable-diffusion-2-1-base

adapter

mit

Views

No views yet

Infarct Image

Model description

InfarctImage - LoRA Fine-Tuned Model for Heart Attack Simulation

📌 Description

InfarctImage is a LoRA-based model fine-tuned on Stable Diffusion 2.1, designed to generate realistic images of people simulating a heart attack. This model was developed as part of a study on synthetic dataset generation for human activity recognition and medical emergency monitoring applications.

🔗 Related Article: Coming soon.

🎯 Objective

The model addresses the issue of data scarcity in medical and anomaly detection environments. Generating high-quality synthetic images enables:

Expanding existing datasets without relying on real-world data.
Overcoming ethical and logistical restrictions in medical image collection.
Enhancing AI-based detection of critical events like heart attacks.

📥 Download and Installation

To use this model with Diffusers, follow these steps:

python

from diffusers import DiffusionPipeline
import torch
# Load the base model
infarct_pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1-base", torch_dtype=torch.float16, use_safetensors=True)
# Load LoRA weights
infarct_pipe.load_lora_weights("Gavit0/InfarctImage")
# Move to GPU if available
infarct_pipe.to(torch.device("cuda" if torch.cuda.is_available() else "cpu"))
print("Model loaded successfully!")

🏆 Usage Examples

You can generate images using prompts like:

python

prompt = ("Elderly man at a sports stadium surrounded by a crowd, "
          "clutching his chest with a distressed look, indicating a heart attack."
         )
negative_prompt = (
          "blurry, deformed face, bad anatomy, poorly drawn face, out of focus, ugly, noisy, extra fingers, "
          "distorted, grainy, worst quality, low quality, low resolution, illustration, "
          "dull, watermark, close-up, 3d, 2d, painting, sketch, render, cartoon, grain, kitsch"
        )
trigger = "Person with expression of pain due to a heart attack, "
full_prompt = f"{trigger}, {prompt}"

image = infarct_pipe(prompt=full_prompt, negative_prompt=negative_prompt,
          guidance_scale=4, num_inference_steps=40).images[0]
image.show()

Full examples in:

🔗 Notebook in GitHub
Open in Google Colab

📊 Training Data

The model was trained on a dataset of 100 manually annotated images, including:

50 images of people simulating heart attack symptoms.
50 images of people in neutral contexts.

The dataset was processed and annotated using BLIP (Bootstrapping Language-Image Pretraining) to enhance image descriptions and improve training prompts.

🔧 Hyperparameters and Configuration

Base Model: Stable Diffusion 2.1
Fine-Tuning Technique: LoRA (Low-Rank Adaptation)
Learning Rate: 0.0001
Batch Size: 1
Epochs: 10
Hardware: NVIDIA RTX 4090 (24GB VRAM)

📈 Model Evaluation

LPIPS (Learned Perceptual Image Patch Similarity) was used to evaluate the quality of the generated images. Results show that LoRA fine-tuning improves the perceptual similarity of generated images compared to real training data.

Model	LPIPS (↓ Better)
SD 2.1 Base	0.7366
SD 2.1 + LoRA	0.6919

📜 License

This model is distributed under the MIT License.

Trigger words

You should use Person with expression of pain due to a heart attack to trigger the image generation.

You should use infarct to trigger the image generation.

Download model

Weights for this model are available in Safetensors format.

Download them in the Files & versions tab.