You can deploy this model and start earning money today!

montignyp/
MYai

Views

No views yet

Flux Identity LoRA Training Environment

A complete training environment for creating identity LoRAs using FLUX.1 Dev on RunPod with NVIDIA A100 80GB.

Overview

This project provides everything needed to:

Train identity-preserving LoRAs on FLUX.1 Dev
Validate and prepare training datasets
Test LoRAs in ComfyUI
Scale to ControlNet and IP-Adapter (future)

Target Configuration

GPU: NVIDIA A100 80GB
Base Model: FLUX.1 Dev
Training Precision: bf16
Image Resolution: 1024x1024
Dataset Size: ~1,000 identity images
Training Toolkit: kohya_ss/sd-scripts

Folder Structure

/workspace/flux-project/
├── models/                    # Model files
│   ├── flux/                  # FLUX.1 Dev base model
│   ├── lora/                  # Trained LoRA outputs
│   ├── vae/                   # VAE (ae.safetensors)
│   ├── clip/                  # Text encoders (CLIP-L, T5-XXL)
│   ├── controlnet/            # ControlNet models (future)
│   └── ip-adapter/            # IP-Adapter models (future)
├── datasets/
│   └── identity/
│       ├── images/            # Raw training images
│       └── processed/         # Processed images for training
├── output/                    # Training outputs & checkpoints
├── scripts/                   # Utility scripts
├── logs/                      # TensorBoard logs
└── configs/                   # Training configurations

Quick Start

1. Initial Setup

Run the setup script on RunPod:

bash

cd /workspace/flux-project/scripts
chmod +x setup_environment.sh
./setup_environment.sh

This will:

Install system dependencies
Create Python virtual environment
Install PyTorch with CUDA support
Install ComfyUI and sd-scripts
Set up model symlinks

2. Download Models

After setup, download the required models:

bash

# Login to Hugging Face
huggingface-cli login

# Download FLUX.1 Dev (requires accepting license)
huggingface-cli download black-forest-labs/FLUX.1-dev \
    --local-dir /workspace/flux-project/models/flux

# Download CLIP-L
huggingface-cli download openai/clip-vit-large-patch14 \
    --local-dir /workspace/flux-project/models/clip/clip-l

# Download T5-XXL
huggingface-cli download google/t5-v1_1-xxl \
    --local-dir /workspace/flux-project/models/clip/t5-xxl

3. Prepare Dataset

Place your images in /workspace/flux-project/datasets/identity/images/

4. Validate Dataset

python /workspace/flux-project/scripts/validate_dataset.py

5. Start Training

/workspace/flux-project/scripts/start_training.sh

Dataset Preparation Workflow

Image Requirements

Requirement	Recommended	Minimum
Resolution	1024x1024	512x512
Format	PNG, JPG	Any supported
Quality	High, uncompressed	No artifacts
Count	500-2000	100
Variety	Multiple angles, lighting, expressions	At least 3 poses

Image Guidelines

Consistency: Same subject across all images
Quality: Sharp, well-lit, no motion blur
Variety: Different:
- Angles (front, 3/4, profile)
- Lighting conditions
- Expressions
- Backgrounds (ideally varied)
- Clothing (for person identity)
Cropping: Subject should fill 50-80% of frame
Resolution: Higher is better (will be resized)

Caption Format

Each image needs a matching .txt file with the same name:

photo_001.jpg  →  photo_001.txt
photo_002.png  →  photo_002.txt

Caption Best Practices

Use a trigger word: Start every caption with a unique identifier

ohwx person, a man with short brown hair, wearing a blue shirt
Be descriptive: Include details about pose, clothing, setting

ohwx person, a woman smiling, professional headshot, studio lighting, grey background
Consistent structure: Keep caption format similar across dataset

[trigger], [subject description], [pose/action], [setting], [lighting]
Avoid:
- Overly long captions (aim for 20-50 words)
- Inconsistent terminology
- Subjective qualities ("beautiful", "amazing")

Trigger Word Strategy

Choose a trigger word that is:

Unique: Not a common word (ohwx, sks, zwx)
Pronounceable: Easy to remember
Consistent: Used identically in every caption

Example trigger words: ohwx, sks, xyz123, [subject]

Running Validation

bash

# Full validation with duplicate detection
python /workspace/flux-project/scripts/validate_dataset.py /workspace/flux-project/datasets/identity/images

# Quick validation (skip duplicates)
python /workspace/flux-project/scripts/validate_dataset.py --no-duplicates /path/to/images

# Save report to file
python /workspace/flux-project/scripts/validate_dataset.py -o report.txt /path/to/images

Dataset Statistics

python /workspace/flux-project/scripts/dataset_stats.py /workspace/flux-project/datasets/identity/images

Training Workflow

Configuration

Edit /workspace/flux-project/configs/train_config.toml:

Key parameters to adjust:

Parameter	Default	Description
`max_train_epochs`	10	Number of training epochs
`learning_rate`	1e-4	Base learning rate
`network_dim`	32	LoRA rank (8-128)
`network_alpha`	16	LoRA alpha (typically dim/2)
`train_batch_size`	1	Images per step
`gradient_accumulation_steps`	4	Effective batch multiplier

Starting Training

bash

# Using default config
/workspace/flux-project/scripts/start_training.sh

# Using custom config
/workspace/flux-project/scripts/start_training.sh /path/to/custom_config.toml

Monitoring Progress

TensorBoard logs are saved to /workspace/flux-project/logs/:

tensorboard --logdir /workspace/flux-project/logs --bind_all --port 6006

Access at: http://<your-runpod-ip>:6006

Checkpoints

Checkpoints are saved to /workspace/flux-project/output/:

identity_lora-epoch-01.safetensors
identity_lora-epoch-02.safetensors
...
identity_lora.safetensors (final)

Recommended Training Schedule

Dataset Size	Epochs	Steps (approx)
100 images	20-30	500-750
500 images	10-15	1250-1875
1000 images	8-12	2000-3000
2000 images	5-8	2500-4000

LoRA Export Workflow

Checkpoint Files

After training, your LoRA files will be in:

/workspace/flux-project/output/
├── identity_lora.safetensors          # Final model
├── identity_lora-epoch-*.safetensors  # Epoch checkpoints
└── training_state/                    # Resume state (if enabled)

Testing Checkpoints

Different epochs may produce different quality results. Test multiple:

Copy checkpoint to LoRA folder:

bash
1cp /workspace/flux-project/output/identity_lora-epoch-05.safetensors \ 2 /workspace/flux-project/models/lora/
Load in ComfyUI and generate test images
Compare quality across epochs

Converting Checkpoints (if needed)

sd-scripts outputs are already in safetensors format. No conversion needed.

Inference Workflow

Starting ComfyUI

/workspace/flux-project/scripts/start_comfyui.sh

Access at: http://<your-runpod-ip>:8188

Loading LoRA in ComfyUI

In ComfyUI, add a "Load LoRA" node
Select your trained LoRA from the dropdown
Connect to your Flux workflow
Set LoRA strength (start with 0.7-1.0)

Recommended Inference Settings

Setting	Value
LoRA Strength	0.7 - 1.0
CFG Scale	3.5 - 7.0
Steps	20 - 30
Sampler	euler, dpmpp_2m
Scheduler	normal, karras

Using Your Trigger Word

Include your trigger word in prompts:

ohwx person, professional portrait photo, studio lighting,
sharp focus, 8k resolution

Experiment with:

Trigger word position (start vs middle)
Combining with style keywords
Different prompt lengths

Future Expansion

ControlNet Integration

Download Flux ControlNet when ready:

bash

# InstantX Flux ControlNet Union
huggingface-cli download InstantX/FLUX.1-dev-Controlnet-Union \
    --local-dir /workspace/flux-project/models/controlnet/flux-controlnet-union

Use cases:

Pose control
Depth guidance
Canny edge control

IP-Adapter Setup

Download IP-Adapter for identity preservation:

bash

huggingface-cli download InstantX/FLUX.1-dev-IP-Adapter \
    --local-dir /workspace/flux-project/models/ip-adapter/flux-ip-adapter

Use cases:

Reference image guidance
Style transfer
Character consistency

Troubleshooting

Common Issues

CUDA Out of Memory

Reduce train_batch_size to 1
Enable gradient_checkpointing
Enable cache_latents_to_disk
Reduce network_dim

Training Loss Not Decreasing

Check caption quality
Try lower learning rate (5e-5)
Verify images are properly formatted
Check for corrupt images with validation script

Blurry/Distorted Results

Train longer (more epochs)
Increase network_dim
Verify dataset quality
Check for duplicate images

ComfyUI Can't Find LoRA

Check symlinks are correct
Verify file is in .safetensors format
Restart ComfyUI

Getting Help

File Reference

File	Purpose
`scripts/setup_environment.sh`	Complete environment setup
`scripts/start_comfyui.sh`	Launch ComfyUI server
`scripts/start_training.sh`	Launch LoRA training
`scripts/validate_dataset.py`	Validate training images
`scripts/dataset_stats.py`	Generate dataset statistics
`configs/train_config.toml`	Training configuration

License

This project setup is provided as-is for educational purposes.

Model licenses:

FLUX.1 Dev: Black Forest Labs License
CLIP: OpenAI License
T5: Apache 2.0

montignyp/MYai

Flux Identity LoRA Training Environment

Overview

Target Configuration

Folder Structure

Quick Start

1. Initial Setup

2. Download Models

3. Prepare Dataset

4. Validate Dataset

5. Start Training

Dataset Preparation Workflow

Image Requirements

Image Guidelines

Caption Format

Caption Best Practices

Trigger Word Strategy

Running Validation

Dataset Statistics

Training Workflow

Configuration

Starting Training

Monitoring Progress

Checkpoints

Recommended Training Schedule

LoRA Export Workflow

Checkpoint Files

Testing Checkpoints

Converting Checkpoints (if needed)

Inference Workflow

Starting ComfyUI

Loading LoRA in ComfyUI

Recommended Inference Settings

Using Your Trigger Word

Future Expansion

ControlNet Integration

IP-Adapter Setup

Troubleshooting

Common Issues

Getting Help

File Reference

License

montignyp/
MYai