FreeMorph / IMPUS MVP

Heads up: Both pipelines are GPU hungry. FreeMorph prefers ≥12GB VRAM; IMPUS often needs 14GB+ plus long per-job fine-tuning runs.

Prerequisites

1. Python 3.10+ and CUDA-compatible PyTorch build (if you plan to use GPU).
2. Install dependencies (pin matches the FreeMorph paper setup):

Running the server (local GPU/CPU)

1. Upload two images (.jpg, .png, etc.). Square crops work best; non-square inputs are center-cropped internally.
2. (Optional) Provide short comma-separated prompts describing each photo.
3. Choose a morphing backend:
   • FreeMorph (SD 2.1) – faster baseline, matches the original paper settings.
   • IMPUS (SD 1.4, ICLR 2024) – perceptually-uniform sampling via textual inversion + LoRA fine-tuning per job. Expect much longer runtimes (tens of minutes) and higher VRAM usage (14GB+ strongly recommended).
4. Click Morph Images and wait for the frames to be generated.

Deploying to Hugging Face Inference Endpoints

1. Login & create repo
   bash

   1huggingface-cli login --token <your_hf_token> 2huggingface-cli repo create zaruta/freemorph-mvp --type=model 3git clone https://huggingface.co/zaruta/freemorph-mvp 4cd freemorph-mvp 5cp -R /Users/zaruta/morph-test/* . 6git add . 7git commit -m "Deploy FreeMorph" 8git push
2. Build & push Docker image (linux/amd64)
   bash

   1cd /Users/zaruta/morph-test 2docker login -u zaruta 3docker buildx build --platform linux/amd64 -t zaruta/freemorph-mvp:latest --push .
3. Create endpoint
   In https://ui.endpoints.huggingface.co/ → New Endpoint:
   • Repository: zaruta/freemorph-mvp
   • Inference Engine: Custom
   • Container URL: docker.io/zaruta/freemorph-mvp:latest
   • Container port: 8000
   • Health route: /api/health
   • Hardware: GPU Medium (A10G), Region: us-east-1
   After the build finishes the endpoint URL looks like https://nziq03no5bepo0w1.us-east-1.aws.endpoints.huggingface.cloud.

Running the UI against the HF endpoint

• The browser always talks to http://localhost:8000.
• FastAPI proxies /api/morph to the remote endpoint, attaching Authorization: Bearer <your_hf_token> when HF_API_TOKEN is set.
• Frames in the JSON response are rewritten to absolute URLs such as https://nziq03no5bepo0w1.us-east-1.aws.endpoints.huggingface.cloud/storage/results/<job>/frame_00.png, so the UI renders images directly from HF storage (локально они не появляются).
• The engine form field is forwarded to the remote endpoint, so hosted deployments expose the same FreeMorph/IMPUS toggle.

Switching between FreeMorph and IMPUS

• The UI dropdown updates the engine form field.
• Locally, FastAPI instantiates the requested service on first use and caches it for subsequent jobs.
• For engine=freemorph, diffusion runs fully in-process via FreeMorphService.
• For engine=impus, the app shells out to the vendored IMPUS reference script (vendor/IMPUS/run_morph.py), waits for it to dump numbered PNG frames, and renames them to the common frame_##.png format consumed by the UI.
• Both engines return the identical JSON contract (jobId, frames, frameCount, durationMs), so the UI and remote deployments stay agnostic.
• The IMPUS files are pulled verbatim from the official repo (MIT License) to stay close to the paper's implementation.

Tips & troubleshooting

• Generation is slow on CPU and may take many minutes per request. For a quick proof-of-concept, switch to a CUDA runtime.
• If VRAM is tight, lower steps, guidance_scale, or interpolation_size in app/freemorph_service.py.
• IMPUS performs textual inversion + LoRA finetuning per job. Expect 30–60 minutes on an A10G/A100 and keep /storage/results/<job> around if you want to inspect the intermediate .pt checkpoints it emits.
• The UI surfaces server errors directly. Inspect the FastAPI logs for PyTorch or diffusion-specific stack traces.

Project layout

• app/ – FastAPI entrypoint plus the FreeMorph/IMPUS service wrappers
• static/ & templates/ – MVP UI assets
• storage/uploads, storage/results – persisted inputs/outputs
• vendor/FreeMorph – untouched upstream FreeMorph implementation
• vendor/IMPUS – upstream IMPUS script + helpers invoked via subprocess

Future ideas

• Async job queue to avoid blocking requests while diffusion runs
• Automatic captioning (e.g., BLIP or Llava) to remove manual prompt entry
• Animated GIF/video export for the generated frames

Deploying to Hugging Face Inference Endpoints

1. Create/choose an HF repo (private is fine) and push this project (Dockerfile must live in the root).
2. In the HF UI go to Endpoints → Create endpoint, pick Custom Docker, select GPU hardware (A10G works well), and point the endpoint at your repo.
3. The build system will run pip install -r requirements.txt and start FastAPI via uvicorn app.main:app --host 0.0.0.0 --port 8000.
4. When the endpoint status becomes Running, your API will be reachable at https://<endpoint>/api/morph; reuse the same JSON contract as locally.