🥖 Baguette
A Distributed Inference Engine for Paris MoE Diffusion Models
Fast, efficient inference for the 5-billion parameter Paris Mixture-of-Experts text-to-image model
⚡ Quick Start
1 # Clone the repo
2 git clone https://huggingface.co/nbagel/baguette
3 cd baguette
4
5 # Install dependencies
6 pip install uv && uv pip install torch torchvision safetensors transformers diffusers accelerate tqdm
7
8 # Generate images
9 python generate.py --prompt "a cute cat" --num_samples 4
Output: output_bf16.png with 4 generated images.
🎨 Generation Examples
1 # Basic generation (4 images, top-2 routing, 30 steps)
2 python generate.py --prompt "sunset over mountains" --num_samples 4
3
4 # See expert routing visualization
5 python generate.py --prompt "abstract art" --visualize
6
7 # Faster generation
8 python generate.py --prompt "a happy dog" --num_steps 20
9
10 # Lower memory usage (offload experts to CPU)
11 python generate.py --prompt "portrait of a scientist" --offload 4
12
13 # INT8 quantized (smaller weights)
14 python generate.py --prompt "enchanted forest" --precision int8
🔮 Expert Routing Visualization
Baguette includes real-time visualization of the MoE router's expert selection. Use --visualize to see which experts are activated:
╭──────────────────────────────────────────────────╮
│ ⚡ EXPERT USAGE DISTRIBUTION │
├──────────────────────────────────────────────────┤
│ → E4 │████████████████████████████│ 40.6% │
│ E2 │██████████████████████████▎ │ 36.7% │
│ E6 │██████████▌ │ 14.8% │
│ E1 │███▊ │ 5.5% │
│ E5 │█▋ │ 2.3% │
│ E0 │ │ 0.0% │
│ E3 │ │ 0.0% │
│ E7 │ │ 0.0% │
├──────────────────────────────────────────────────┤
│ Active: 5/8 experts Calls: 128 │
╰──────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────╮
│ 📈 ROUTING TIMELINE │
├──────────────────────────────────────────────────┤
│ Step 0 1 2 3 4 5 6 7 8 9 10 11 12 13 │
│ ─────────────────────────────────────────────── │
│ E0 · · · · · · · · · · · · · · │
│ E1 · · · · · · · · · · · · · · │
│ E2 · · · · · ● ● ● ● ● ● ● ● ● │
│ E3 · · · · · · · · · · · · · · │
│ E4 · · ● ● ● · · · · · · · · · │
│ E5 · · · · · · · · · · · · · · │
│ E6 ● ● · · · · · · · · · · · · │
│ E7 · · · · · · · · · · · · · · │
├──────────────────────────────────────────────────┤
│ Routing changes: 2/13 steps (15%) │
╰──────────────────────────────────────────────────╯The router dynamically selects different experts based on the noise level at each diffusion timestep. Early steps (high noise) often use different experts than later steps (low noise).
📋 Command Reference
Flag Default Description --prompt"a cute cat"Text description of the image to generate --num_samples16Number of images to generate --num_steps30Diffusion sampling steps (15-50) --cfg_scale7.5Classifier-free guidance scale (5-12) --precisionbf16Weight precision: bf16 or int8 --topk2Number of experts per sample (1-8) --offload0Experts to offload to CPU RAM (0-7) --visualizefalseShow expert routing statistics --outputautoCustom output filename --seed999Random seed for reproducibility
🏗️ Model Architecture
┌─────────────────────────────────────────────────────────────────┐
│ PARIS MoE ARCHITECTURE │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Input: Text Prompt ──→ CLIP ViT-L/14 ──→ Text Embeddings │
│ │
│ Noise: z ~ N(0,1) ──→ 32×32×4 Latent │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ DiT-B/2 ROUTER │ │
│ │ (12 layers, 768 dim, 129M params) │ │
│ │ │ │ │
│ │ Selects Top-K Experts per Step │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────┼───────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
│ │ Expert 0 │ │ Expert 1 │ ··· │ Expert 7 │ │
│ │ DiT-XL/2 │ │ DiT-XL/2 │ │ DiT-XL/2 │ │
│ │ 606M │ │ 606M │ │ 606M │ │
│ └────────────┘ └────────────┘ └────────────┘ │
│ │ │ │ │
│ └───────────────────┼───────────────────┘ │
│ ▼ │
│ Weighted Velocity Prediction │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ SD-VAE DECODER │ │
│ │ Latent ──→ 256×256 RGB │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
├─────────────────────────────────────────────────────────────────┤
│ Total: ~5 Billion Parameters │ 8 Specialized Experts │
└─────────────────────────────────────────────────────────────────┘💾 Available Weights
Format Size Quality Speed Use Case BF16 9.3 GB ⭐⭐⭐⭐⭐ Fastest Production, best quality INT8 4.8 GB ⭐⭐⭐⭐ Fast Memory-constrained GPUs
🖥️ Memory Requirements
Configuration GPU VRAM Speed Notes BF16, no offload ~25 GB ~3 img/s Best performance BF16, offload 4 ~14 GB ~1 img/s RTX 4090 / A6000 BF16, offload 6 ~8 GB ~0.5 img/s RTX 3080/4080 INT8, no offload ~12 GB ~2 img/s Good balance INT8, offload 4 ~8 GB ~0.5 img/s Consumer GPUs
🔧 Utilities
Benchmarking
1 python benchmark.py --quick # Fast benchmark
2 python benchmark.py --output results.md # Full benchmark, save results
Weight Conversion
1 # Convert PyTorch checkpoints to BF16 SafeTensors
2 python quantize.py --input /path/to/weights --output ./weights/bf16 --format bf16
3
4 # Convert BF16 to INT8
5 python quantize.py --input ./weights/bf16 --output ./weights/int8 --format int8
🚀 Future: Distributed Inference with Tailscale + Erlang
Baguette is being developed as a
fully distributed inference engine that can run across multiple machines connected via
Tailscale VPN, orchestrated by an Erlang/OTP supervisor.
🌐 Architecture Vision
┌─────────────────────────────────────────────────────────────────────────┐
│ BAGUETTE DISTRIBUTED NETWORK │
│ (Up to 8 Nodes) │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ Tailscale VPN Mesh ┌─────────────┐ │
│ │ Node 1 │◄────────────────────────────►│ Node 2 │ │
│ │ ┌─────────┐ │ │ ┌─────────┐ │ │
│ │ │ Router │ │ │ │ Router │ │ │
│ │ │ VAE │ │ │ │ VAE │ │ │
│ │ │Expert 0 │ │ │ │Expert 1 │ │ │
│ │ └─────────┘ │ │ └─────────┘ │ │
│ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │
│ │ ┌──────────────────┐ │ │
│ └────────►│ Erlang/OTP │◄─────────────┘ │
│ │ Coordinator │ │
│ ┌────────►│ │◄─────────────┐ │
│ │ │ • Load Balance │ │ │
│ │ │ • Fault Tolerant│ │ │
│ │ │ • Auto-Healing │ │ │
│ │ └──────────────────┘ │ │
│ │ │ │
│ ┌──────┴──────┐ ┌──────┴──────┐ │
│ │ Node 3 │◄────────────────────────────►│ Node 4 │ │
│ │ ┌─────────┐ │ ... │ ┌─────────┐ │ │
│ │ │ Router │ │ │ │ Router │ │ │
│ │ │ VAE │ │ (up to 8 nodes) │ │ VAE │ │ │
│ │ │Expert 2 │ │ │ │Expert 3 │ │ │
│ │ └─────────┘ │ │ └─────────┘ │ │
│ └─────────────┘ └─────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘🎯 Key Features (Planned)
Feature Description Self-Organizing Network Nodes automatically discover peers and negotiate roles Adaptive Load Balancing Routes requests based on real-time latency and compute availability Auto-Benchmarking Each node benchmarks GPU/CPU speed, VRAM, RAM, and network throughput Fault Tolerance Erlang supervisors restart failed nodes, redistribute load 1 Expert Per Node Each node loads only 1 expert (~2.7GB VRAM) plus router & VAE Latency-Aware Routing Prioritizes low-latency nodes for time-sensitive steps Zero Configuration Just join the Tailscale network and run—automatic peer discovery
📊 Node Self-Benchmarking
When a node joins the network, it automatically benchmarks:
┌────────────────────────────────────────┐
│ NODE CAPABILITY REPORT │
├────────────────────────────────────────┤
│ GPU: NVIDIA RTX 4090 │
│ VRAM: 24 GB │
│ GPU Compute: 847 TFLOPS (FP16) │
│ ──────────────────────────────────── │
│ CPU: AMD Ryzen 9 7950X │
│ RAM: 64 GB │
│ CPU Compute: 2.1 TFLOPS │
│ ──────────────────────────────────── │
│ Network Latency to Peers: │
│ → Node 2: 12ms │
│ → Node 3: 8ms │
│ → Node 4: 45ms │
│ Network Bandwidth: 940 Mbps │
│ ──────────────────────────────────── │
│ Assigned Expert: E0 │
│ Status: READY │
└────────────────────────────────────────┘🔄 Distributed Inference Flow
Request arrives at any nodeRouter runs locally → selects top-K experts neededCoordinator dispatches expert calls to appropriate nodesNodes compute in parallel → return velocity predictionsResults aggregated → Euler step appliedVAE decodes locally → image returned to requester
This enables running the full 5B parameter model across consumer hardware—each machine only needs ~4GB VRAM to hold one expert.
📁 Repository Structure
baguette/
├── generate.py # 🎨 Main generation script
├── benchmark.py # 📊 Performance benchmarking
├── quantize.py # 🔧 Weight format conversion
├── requirements.txt # 📦 Python dependencies
├── README.md # 📖 This file
├── src/ # 🧠 Model architecture code
│ ├── models.py # DiT expert & router definitions
│ ├── vae_utils.py # VAE encoding/decoding
│ ├── config.py # Configuration dataclass
│ └── schedules.py # Noise schedules
└── weights/ # 💾 Model weights
├── bf16/ # BFloat16 SafeTensors (9.3 GB)
│ ├── expert_0.safetensors ... expert_7.safetensors
│ ├── router.safetensors
│ └── config.pt
└── int8/ # INT8 Quantized (4.8 GB)
├── expert_0.safetensors ... expert_7.safetensors
└── router.safetensors🔗 Links
Original Model : bageldotcom/paris This Repository : nbagel/baguette
📜 License
This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) .
Made with 🥖 by the Baguette Team
Distributed inference for everyone
