You can deploy this model and start earning money today!

mmahdi-sz/
FeyNobg-ONNX

Name: FeyNobg-ONNX
Availability: InStock

Views

No views yet

FeyNoBg — ONNX CPU Inference Benchmark

Optimized ONNX export of FeyNoBg (BiRefNet-based background removal model), benchmarked exhaustively on CPU with FP32, FP16, and Dynamic INT8 quantization.

What's in this repo?

File	Description	Size
`feynobg_fp32.onnx`	Full precision ONNX export	~1.1 GB
`feynobg_fp16.onnx`	Half precision export	~556 MB
`feynobg_int8_dynamic.onnx`	Dynamic INT8 quantized (recommended for CPU)	~1.3 GB

🎨 Inference Examples

🚀 Quick Start

Install dependencies

bash

git clone https://huggingface.co/mmahdi-sz/FeyNobg-ONNX
cd FeyNobg-ONNX
pip install onnxruntime numpy pillow

Run inference (ONNX — recommended for CPU deployment)

python

import numpy as np
import onnxruntime as ort
from PIL import Image

# ── Load model ──────────────────────────────────────────────────────────────
# Recommended: Dynamic INT8 + 8 cores for the best latency/cost ratio
ONNX_PATH = "feynobg_int8_dynamic.onnx"
NUM_CORES  = 8   # adjust to your server's core count

sess_opts = ort.SessionOptions()
sess_opts.intra_op_num_threads = NUM_CORES
sess_opts.inter_op_num_threads = 1
sess_opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL

session = ort.InferenceSession(
    ONNX_PATH,
    sess_options=sess_opts,
    providers=["CPUExecutionProvider"]
)

# ── Preprocess ──────────────────────────────────────────────────────────────
def preprocess(image_path: str, size=(1024, 1024)) -> np.ndarray:
    img = Image.open(image_path).convert("RGB").resize(size)
    x   = np.array(img, dtype=np.float32) / 255.0
    mean = np.array([0.485, 0.456, 0.406])
    std  = np.array([0.229, 0.224, 0.225])
    x    = (x - mean) / std
    return x.transpose(2, 0, 1)[np.newaxis].astype(np.float32)  # [1, 3, H, W]

# ── Inference ───────────────────────────────────────────────────────────────
input_tensor = preprocess("your_image.jpg")
input_name   = session.get_inputs()[0].name

alpha_mask = session.run(None, {input_name: input_tensor})[0]  # [1, 1, H, W]
mask = (alpha_mask[0, 0] * 255).clip(0, 255).astype(np.uint8)
Image.fromarray(mask).save("output_mask.png")
print("Done! Mask saved to output_mask.png")

Run inference (PyTorch — original model)

pip install nobg torch

python

import nobg
import torch
from PIL import Image

model = nobg.AutoModel.from_pretrained("nobg/FeyNobg")
model.eval()

# ... standard PyTorch inference

Note: PyTorch (16 cores, no pinning) averages 40,743ms per image. The ONNX INT8 model on 8 pinned cores achieves 11,165ms — 3.6× faster.

📊 CPU Benchmark Results

All benchmarks run on an AMD EPYC-Genoa Processor (16 cores), images resized to 1024×1024, with cores strictly pinned using taskset (no OS interference).

10 test images per configuration
1 warmup run discarded per image
Latency = mean of timed inference runs only

Detailed results: 4 / 8 / 12 Cores

Model Variant	Optimization	4 Cores	8 Cores	12 Cores	IoU ↑	MAE ↓
FP32	ORT_ENABLE_ALL	17,849ms	11,971ms	14,325ms	1.000	0.0004
FP32	ORT_ENABLE_BASIC	18,230ms	12,723ms	15,760ms	1.000	0.0004
FP16	ORT_ENABLE_ALL	19,156ms	13,382ms	16,966ms	1.000	0.0004
FP16	ORT_ENABLE_BASIC	19,154ms	13,218ms	16,870ms	1.000	0.0004
Dynamic INT8	ORT_ENABLE_ALL	15,868ms	11,165ms	14,568ms	1.000	0.0004
Dynamic INT8	ORT_ENABLE_BASIC	16,938ms	12,876ms	15,706ms	1.000	0.0004
PyTorch FP32 (baseline, 16 cores)	—	—	—	40,743ms	1.000	0.0004

✅ Zero quality degradation. All ONNX variants (FP32, FP16, Dynamic INT8) achieve IoU = 1.000 and MAE = 0.0004 compared to ground-truth masks — identical to the original PyTorch model.

📈 CPU Core Scaling — Dynamic INT8 (1 to 14 Cores)

Cores	Latency	Speedup vs PyTorch	Notes
1	45,300ms	0.9×	—
2	26,101ms	1.6×	—
4	15,897ms	2.6×	—
6	12,577ms	3.2×	—
8	11,165ms	3.6×	⭐ Value Sweet Spot
10	9,936ms	4.1×	—
12	9,350ms	4.4×	🏆 Max Performance
14	9,359ms	4.4×	Plateau — no further gain

Observations:

Scaling is near-linear up to ~8 cores, then hits a RAM bandwidth bottleneck.
After 12 cores, latency plateaus — additional cores provide no benefit for this model at 1024×1024.
FP16 is slower than FP32 on CPU (x86 CPUs lack native FP16 compute units).
Dynamic INT8 leverages AVX2/AVX-512 integer instructions for maximum throughput.

🔑 Key Findings

Finding	Detail
🏆 Best overall	`Dynamic INT8` + `ORT_ENABLE_ALL` + 8 cores → 11.2s
⚡ Absolute fastest	`Dynamic INT8` + 12 pinned cores → 9.35s
🔬 Quality	All ONNX variants: IoU = 1.000, MAE = 0.0004 (no degradation)
🐢 FP16 on CPU	Slower than FP32 (CPU lacks native FP16 math units)
📉 PyTorch vs ONNX INT8	40.7s → 9.4s = 4.4× speedup at peak
🧠 RAM bottleneck	Plateau after 12 cores — memory-bound workload

🔁 Reproduce the Benchmark

bash

git clone https://huggingface.co/mmahdi-sz/FeyNobg-ONNX
cd FeyNobg-ONNX

# Create virtualenv
python -m venv venv && source venv/bin/activate
pip install onnxruntime numpy pillow

# Run scaling benchmark (1–14 cores, generates CSV + chart)
python hf_scaling_benchmark.py

# Run full 4/8/12 core benchmark (all variants)
python run_benchmark.py

All benchmarks use taskset -c 0-<N-1> to strictly pin CPU cores at the OS level, preventing thread migration and ensuring reproducible measurements.

Hardware Tested

Property	Value
CPU	AMD EPYC-Genoa Processor (16-core)
RAM	DDR5 (Sufficient for 1.3GB model + inference buffers)
OS	Linux (Ubuntu)
ONNX Runtime	Latest stable
Python	3.13
Input resolution	1024 × 1024

Citation

If you use this benchmark in your work, please cite the original FeyNoBg model:

@model{feynobg,
  author = {nobg},
  title  = {FeyNoBg: BiRefNet-based Background Removal},
  year   = {2024},
  url    = {https://huggingface.co/nobg/FeyNobg}
}

Benchmarks by @mmahdi-sz