Indic Heritage Studio v2
A multimodal content creation tool that reimagines modern photos and prompts through the lens of Indian heritage art forms — running on SDXL + Stable Video Diffusion + ControlNet + per-style LoRAs across 8 × NVIDIA 80GB GPUs (dev) and AMD Radeon Cloud (final demo).
Submission for the AMD AI DevMaster Hackathon (July 15 – August 6, 2026)
Track 1: Multimodal AI — Development of Multimodal Content Creation Tools
Team: TeamIndicForge
🎯 What It Does (v2 — full rebuild)
Indic Heritage Studio v2 transforms everyday inputs into culturally-rooted visual art. Given a text prompt, a reference photo, or both, it produces:
Text → Heritage-styled image — SDXL 1.0 (DreamShaper-XL turbo) at 1024² with per-style LoRA overlayImage → Stylized image — IP-Adapter XL on SDXL with optional multi-style blending (60% Madhubani + 40% Warli)Image → 4-second video — Stable Video Diffusion XT 1.1 (25 frames @ 8 fps, 1024×576)ControlNet composition — Canny / Depth / OpenPose conditioning for precise composition controlInpainting — mask a region, restyle it in any heritage style (heritage restoration use case)Batch processing — multi-GPU data parallelism across 4 worker GPUs (~4× speedup)
Each output is styled after one of five authentic Indian art traditions, each with its own fine-tuned LoRA trained on real heritage art samples:
🟢 Madhubani — Bihar folk art, geometric patterns, natural pigments
🟤 Warli — Maharashtra tribal art, white-on-earth, stick figures
🔵 Pattachitra — Odisha cloth painting, mythological narratives
🟡 Mughal Miniature — court painting, fine detail, gold leaf
🔴 Tanjore — Tamil Nadu, gold-gilded devotional icons
An optional AI Style Advisor (powered by AMD's free Qwen API) helps users pick a style and refine prompts — but the core generation runs 100% on AMD Radeon GPU in compliance with Track 1 rules.
🆕 What's New in v2
Axis v1 v2 T2I model SD 1.5 (DreamShaper) SDXL 1.0 + DreamShaper-XL turbo Resolution 512×512 1024×1024 (1536² for showcase) Style conditioning Prompt tags only Per-style LoRA fine-tune + IP-Adapter XL Image → video AnimateDiff 16f Stable Video Diffusion XT 1.1, 25 frames Composition control ❌ ControlNet (Canny/Depth/OpenPose) Image repair ❌ SDXL inpainting Batch Single GPU Multi-GPU data parallel, 4 workers Pipelines Load/unload on demand All 4 resident on dedicated GPUs
🏗️ System Architecture (Multi-GPU)
┌─────────────────────────────────────────────────────────────────┐
│ USER INTERFACE (Gradio) │
│ 6 tabs + Style Advisor widget + GPU monitor sidebar │
└──────────────────────────────────┬──────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────────┐
│ AGENT LAYER (free AMD Qwen/DeepSeek API) │
│ StyleAdvisor · PromptEngineer · Critic │
└──────────────────────────────────┬──────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────────┐
│ CORE GPU LAYER — MULTI-GPU (8 × 80 GB dev) │
│ GPU 0: SDXL T2I + Inpainting GPU 4: Batch worker 0 │
│ GPU 1: SDXL + IP-Adapter XL GPU 5: Batch worker 1 │
│ GPU 2: SVD-XT 1.1 (I2V) GPU 6: Batch worker 2 │
│ GPU 3: SDXL + ControlNet GPU 7: Batch worker 3 │
│ Per-style LoRAs (~150 MB each) loaded on demand │
└─────────────────────────────────────────────────────────────────┘On AMD Radeon Cloud (single GPU), all pipelines share GPU 0 and load/unload on demand. The code auto-detects device count.
🛠️ Tech Stack
Layer Tech Dev hardware 8 × NVIDIA A100/H100 80GB (640 GB total VRAM) Final demo hardware AMD Radeon Cloud (Radeon GPU, ~16 GB VRAM) Software stack ROCm 6.2 (AMD) / CUDA 12.1 (NVIDIA dev) Deep learning PyTorch 2.4.1 + Diffusers 0.30 + Accelerate 0.34 + PEFT 0.12 T2I model SDXL 1.0 + DreamShaper-XL turbo Style transfer IP-Adapter XL (h94/IP-Adapter) Image-to-video Stable Video Diffusion XT 1.1 (stabilityai/stable-video-diffusion-img2vid-xt-1-1) Composition control ControlNet (Canny/Depth/OpenPose SDXL variants) Per-style LoRAs Trained with PEFT, rank 32, ~800 steps each Agent LLM Free AMD Model APIs (Qwen3.6-35B-A3B / DeepSeek-V4-Flash) UI Gradio 4.x Video post FFmpeg + OpenCV + imageio Image post Pillow, imageio
📦 Installation
Step 1 — Clone
1 git clone https://huggingface.co/Dev2506/indic-heritage-studio
2 cd indic-heritage-studio
Setup
python -m venv .venv
source .venv/bin/activate
pip install torch==2.4.1 torchvision==0.19.1 --index-url
https://download.pytorch.org/whl/cu121
pip install -r requirements.txt
Configure
cp .env.example .env
Launch
python app.py
```
🚀 Usage
CLI mode
1 # Text-to-image in Madhubani style (SDXL + LoRA)
2 python -m core.text_to_image \
3 --prompt "a young woman reading under a banyan tree" \
4 --style madhubani \
5 --out outputs/t2i_madhubani.png \
6 --steps 25 --size 1024 --seed 42
7
8 # Style transfer (IP-Adapter XL)
9 python -m core.style_transfer \
10 --image inputs/photo.jpg \
11 --style warli \
12 --out outputs/styled_warli.png \
13 --strength 0.7 --ip-scale 0.7
14
15 # Image-to-video (SVD)
16 python -m core.image_to_video \
17 --image outputs/styled_warli.png \
18 --out outputs/warli_animated.mp4 \
19 --frames 25 --fps 8
20
21 # ControlNet (Canny → Mughal)
22 python -m core.controlnet \
23 --condition-image inputs/sketch.png \
24 --condition-type canny \
25 --prompt "a courtly gathering" \
26 --style mughal \
27 --out outputs/controlnet_mughal.png
28
29 # Inpainting
30 python -m core.inpainting \
31 --image inputs/damaged_painting.jpg \
32 --mask inputs/damage_mask.png \
33 --prompt "restore the floral border" \
34 --style tanjore \
35 --out outputs/restored.png
36
37 # Batch (multi-GPU, 4 workers)
38 python -m core.batch_processor \
39 --mode style_transfer \
40 --input-dir inputs/ \
41 --output-dir outputs/batch/ \
42 --workers 4
Web UI
The UI exposes all 6 modes plus the Style Advisor widget and GPU monitor.
⚡ Multi-GPU Optimization
The codebase applies these optimizations:
Pipeline-to-GPU pinning — T2I on GPU 0, IP-Adapter on GPU 1, SVD on GPU 2, ControlNet on GPU 3. No thrash.Batch data parallelism — Inputs sharded across GPUs 4-7 via multiprocessing.spawn.Per-style LoRA hot-swap — unload_lora_weights → load_lora_weights in ~3 seconds (vs 30+ sec full pipeline reload).FP16 inference — torch_dtype=torch.float16 everywhere.SDPA attention — built into PyTorch 2.0+, replaces xformers (CUDA-only).Attention + VAE slicing — reduces VRAM peak ~30%.torch.inference_mode()Pipeline singleton reuse — load once, serve many requests.
Multi-GPU Benchmark
python scripts/benchmark.py --configs 1 2 4 8 --samples 4Reports: latency per image, throughput (img/min), peak VRAM, scaling chart.
📊 Evaluation Rubric Coverage
Rubric item Points How v2 hits it Complete input→processing→output workflow 40 6 pipelines: T2I, I2I-style, I2V, ControlNet, Inpaint, batch Innovative creation scenarios 20 Indian heritage art × LoRA fine-tuning × ControlNet × SVD = unique vertical Practical application & social value 20 Cultural preservation + heritage restoration + accessible creator tool Clear, stable, diverse output on Radeon GPU 20 6 output modes × 5 art styles × LoRA = 30+ high-quality demo outputs
🗂️ Project Structure
indic-heritage-studio-v2/
├── README.md # This file
├── app.py # Gradio UI entry point
├── requirements.txt # Python dependencies
├── .env.example # Environment template
├── .gitignore
├── config/
│ ├── settings.py # v2 global config (SDXL, multi-GPU, LoRA paths)
│ └── styles.py # 5 heritage styles + LoRA scales + motion tuning
├── agents/ # LLM agent layer (free AMD API)
│ ├── base.py
│ ├── style_advisor.py
│ ├── prompt_engineer.py # SDXL-aware prompt enrichment
│ └── critic.py # Heuristic + LLM critique
├── core/ # GPU layer
│ ├── text_to_image.py # SDXL + LoRA + optional refiner
│ ├── style_transfer.py # IP-Adapter XL + multi-style blending
│ ├── image_to_video.py # Stable Video Diffusion XT 1.1
│ ├── controlnet.py # Canny / Depth / OpenPose
│ ├── inpainting.py # SDXL inpainting + LoRA
│ └── batch_processor.py # Multi-GPU data parallel
├── training/ # NEW — LoRA training
│ ├── prepare_dataset.py # Heritage art dataset prep
│ └── train_lora.py # PEFT LoRA on SDXL UNet
├── ui/
│ └── gradio_app.py # 6-tab web UI
├── utils/
│ ├── gpu_utils.py # Multi-GPU device management
│ ├── image_utils.py
│ └── video_utils.py
├── scripts/
│ ├── benchmark.py # Multi-GPU scaling benchmark
│ ├── download_models.py # All v2 models (~35 GB)
│ ├── generate_demo_outputs.py # Pre-bake demo gallery
│ ├── verify_rocm.py # Env health check
│ ├── day1_setup.sh # Day 1 environment setup
│ └── amd_demo_recording.sh # AMD demo recording script
├── assets/
│ ├── styles/ # IP-Adapter reference images (5)
│ ├── datasets/ # LoRA training datasets
│ └── loras/ # Trained LoRA weights (5 × ~150 MB)
├── docs/
│ ├── architecture.md # Detailed v2 architecture
│ ├── 2_week_plan.md # Replan for 8×80GB dev
│ ├── local_dev_setup.md # NVIDIA CUDA setup
│ └── architecture_diagram.png
├── examples/ # Demo gallery (pre-baked)
└── tests/ # Smoke tests📅 Build Schedule (2-week plan)
See
docs/2_week_plan.md for the day-by-day plan. Summary:
Week 1 (Jul 16–22): Env setup, source heritage art datasets, train 5 LoRAs, smoke-test all 6 pipelines.Week 2 (Jul 23–29): Pre-bake demo gallery, run multi-GPU benchmark, write PDF + PPT, code freeze.Week 3 (Jul 30–Aug 5): AMD verification + demo recording + submit. Burns ~1.5 of 10 AMD credits.
📝 Submission Package
The 4 required deliverables (Track 1):
Project Profile Document (PDF) — generated via the pdf skill in Week 2Source Code (this repo) — fork of AMD-DEV-CONTEST/Radeon-hackathon-2026-07Demo Video (3–5 min) — rocm-smi → CLI → live GPU inference → UI walkthroughPPT / Poster — generated via the pptx skill in Week 2
PR title format: Track 1, TeamIndicForge, Indic Heritage Studio
🆘 Troubleshooting
Issue Fix torch.cuda.is_available() == FalseReinstall PyTorch with --index-url https://download.pytorch.org/whl/cu121 LoRA training OOMs Lower --rank from 32 → 16, or --batch-size from 1 → 1 with --grad-accum 8 SVD OOM on AMD Use --frames 14 (instead of 25) ControlNet preprocessor download fails python -c "from controlnet_aux import CannyDetector; CannyDetector.from_pretrained('lllyasviel/Annotators')"Multi-GPU batch fails to spawn Set MULTI_GPU=0 in .env HuggingFace download slow HF_HUB_ENABLE_HF_TRANSFER=1 (already in .env)Qwen API 429 errors Agent layer is optional; core tool still works
📧 Contact
📜 License
MIT for project code. Model checkpoints retain their original licenses (see HuggingFace model cards).
