You can deploy this model and start earning money today!

kkytai/
flux2-dev-handler

endpoints_compatible

Views

No views yet

FLUX.2-dev Handler for HuggingFace Dedicated Endpoints

Image generation using FLUX.2-dev with Remote Text Encoding for HuggingFace Dedicated Endpoints.

Architecture

This handler uses Remote Text Encoding to work around PyTorch version limitations on HuggingFace Inference Endpoints:

HF Endpoints runtime: PyTorch 2.2.0, transformers 4.48.0
FLUX.2 Mistral3 text encoder requires: transformers 4.52+ (PyTorch 2.6+)
Solution: Use HuggingFace's remote text encoding API

Benefits:

Saves ~24GB VRAM (no local Mistral3 text encoder)
Works with HF Endpoints' PyTorch 2.2.0 runtime
BF16 precision for transformer and VAE

Features

Remote text encoding via HuggingFace API
Multi-reference image support (up to 10 images)
1920x1080 output resolution by default

GPU Requirements

GPU	VRAM	Mode	Notes
H200	141GB	Full GPU	Recommended
H100	80GB	Full GPU	Good
A100	80GB	CPU offload	Minimum

Deployment Steps

1. Create HuggingFace Repository

bash

# Install huggingface_hub CLI
pip install huggingface_hub

# Login
huggingface-cli login

# Create repository
huggingface-cli repo create flux2-dev-handler --type model

2. Push Handler Files

bash

cd huggingface-endpoint/flux2-dev-handler

# Clone the repo
git clone https://huggingface.co/YOUR_USERNAME/flux2-dev-handler
cd flux2-dev-handler

# Copy handler files
cp ../handler.py .
cp ../requirements.txt .

# Commit and push
git add .
git commit -m "Add FLUX.2-dev handler with remote text encoding"
git push

3. Create Dedicated Endpoint

Go to HuggingFace Endpoints
Click "New Endpoint"
Configure:
- Repository: YOUR_USERNAME/flux2-dev-handler
- Cloud Provider: AWS or Azure
- Region: Your preferred region
- Instance Type: nvidia-h200-141gb (recommended) or nvidia-h100-80gb
- Scaling: Min 0, Max 1 (scale-to-zero enabled)
- Scale-to-Zero Timeout: 15 minutes
Environment Variables:
- HF_TOKEN: Your HuggingFace token (REQUIRED - for model access AND remote text encoding)
- NUM_INFERENCE_STEPS: 50 (optional, default is 50)
Click "Create Endpoint"

4. Wait for Deployment

First deployment takes 5-10 minutes to:

Download FLUX.2-dev transformer and VAE (~35GB without text encoder)
Test remote text encoding connection

5. Test the Endpoint

bash

# Health check (via HuggingFace API)
curl "https://YOUR_ENDPOINT_URL" \
  -H "Authorization: Bearer YOUR_HF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": "test",
    "parameters": {"mode": "generate", "num_inference_steps": 2, "width": 512, "height": 512}
  }'

API Reference

Generate Image

Request:

json

{
  "inputs": "A beautiful sunset over mountains, photorealistic, 16:9",
  "parameters": {
    "mode": "generate",
    "width": 1920,
    "height": 1080,
    "num_inference_steps": 50,
    "guidance_scale": 4.0,
    "seed": -1
  }
}

Response:

json

{
  "image": "data:image/png;base64,...",
  "generation_time_seconds": 12.5
}

Generate with Reference Images

Request:

json

{
  "inputs": "the person from image 1 standing in a park, sunset lighting",
  "parameters": {
    "mode": "generate_with_refs",
    "reference_images": [
      "data:image/png;base64,..."
    ],
    "width": 1920,
    "height": 1080,
    "num_inference_steps": 50,
    "guidance_scale": 2.5,
    "seed": -1
  }
}

Response:

json

{
  "image": "data:image/png;base64,...",
  "generation_time_seconds": 15.2,
  "reference_count": 1
}

Environment Variables

Variable	Default	Description
`HF_TOKEN`	-	HuggingFace token (REQUIRED for model access and remote text encoding)
`NUM_INFERENCE_STEPS`	`50`	Default inference steps

Note: Unlike the Runpod version, this handler does NOT support FP8 quantization or torch.compile due to HF Endpoints' PyTorch 2.2.0 runtime.

Troubleshooting

Endpoint fails to start

Check HF_TOKEN has access to FLUX.2-dev (gated model)
Ensure HF_TOKEN is valid for remote text encoding API
Ensure GPU instance type has sufficient VRAM (80GB+)

"module 'torch' has no attribute 'int1'" error

This means transformers was upgraded beyond what's compatible with PyTorch 2.2.0. Do NOT add transformers>=4.47.0 to requirements.txt.

Remote text encoding fails

Verify HF_TOKEN is set and valid
Check network connectivity to remote-text-encoder-flux-2.huggingface.co

Slow cold start

First inference after scale-up takes 2-3 minutes
Model loading is the main bottleneck

Out of memory

Use H200 for best memory headroom
Reduce image resolution if needed

License

MIT

kkytai/flux2-dev-handler

FLUX.2-dev Handler for HuggingFace Dedicated Endpoints

Architecture

Features

GPU Requirements

Deployment Steps

1. Create HuggingFace Repository

2. Push Handler Files

3. Create Dedicated Endpoint

4. Wait for Deployment

5. Test the Endpoint

API Reference

Generate Image

Generate with Reference Images

Environment Variables

Troubleshooting

Endpoint fails to start

"module 'torch' has no attribute 'int1'" error

Remote text encoding fails

Slow cold start

Out of memory

License

kkytai/
flux2-dev-handler