Model Card for Model ID
This model is a finetuned LoRA on top of Stable Diffusion which aims to enhance accuracy of yoga poses. The finetuned model is trained on 107 classes of yoga poses spread over 7k+ images.
Model Description
- Developed by: [hari106]
- Model type: [Adapter]
- License: [openrail]
Uses
Generates better and more accurate yoga poses than base SD 1.5
Out-of-Scope Use
While the dataset covered a plethora of popular poses, there is a chance some of them might be missed out - esoteric or obscure yoga poses.
Bias, Risks, and Limitations
The model performs well on standard and simple poses, but can get jittery with complex inversions or spinal twists such as "yoganidrasana". The captioning quality is also not the best in this case, and could need improvements. Simple keywords like "Adho Mukha Svanasana, Downward facing dog, yoga pose, full body" helps get better results.
Training Details
Training Data
Training Hyperparameters
- Training regime: fp16 mixed precision
- Epochs: 10
- Optimizer: AdamW
- Learning Rate: 1e-4
- UNet Learning Rate: 1e-4
- Scheduler:: cosine_with_restarts
- Network Module: networks.lora
- Network Dimensions: 32
- Network Alpha: 16
Metrics
Rather than FID and CLIP which was not really the evaluation criteria, the images were evaluated on the basis of joint angle statistics.
Namely:
- Object Keypoint Similarity (OKS)
- Percentage of Correct Keypoints (PCK)
- Joint Angle Error (JAE)
Results
The evaluation CSV could be found in the repo. Check [summary_geometry.csv]
Summary
The model performs quite well with JAE metric compared to the base model. Roughly 81 classes or yoga poses had better JAE score as compared to the reference images than the base model.
Model Card Contact
