Lucida-fp16 (MLX)
egeorcun/lucida converted for
MLX / MLX-Swift on Apple
Silicon. Lucida is a fine-tune of
ZhengPeng7/BiRefNet_HR
for foreground matting / background removal, tuned toward the cases most open matting models handle
worst: camouflaged subjects, transparency, text and logos, print design, and illustration.
Single tier at 1024×1024, the resolution Lucida was trained at.
What was converted
model.safetensors — 687 tensors,
fp16 (~440 MB). Produced from the upstream fp32 checkpoint with
the
birefnet-convert tool in
xocialize/mlx-birefnet-swift:
- checkpoint prefixes stripped; PyTorch-only buffers dropped (
relative_position_index,
num_batches_tracked — 67 tensors)
decoder.conv_out1.0.{weight,bias} → decoder.conv_out1.{weight,bias} (upstream wraps the final
conv in an nn.Sequential; the Swift port uses a bare conv)- every 4-D conv weight transposed from PyTorch NCHW
(O, I, kH, kW) to MLX NHWC (O, kH, kW, I)
- cast to fp16
The architecture is unchanged from BiRefNet_HR — the converted key set is byte-identical to both
upstream BiRefNet bases (754 tensors in → 687 out, zero keys or shapes differing), so this loads
through the stock BiRefNet graph (swin_v1_l, mul_scl_ipt=cat, cxt_num=3, ASPPDeformable).
Verification
Gated against a PyTorch oracle at 1024², with the oracle's exact normalized input tensor injected on
both sides (so the comparison measures the model, not preprocessing):
| |
|---|
| encoder stages (CPU stream) | maxAbs ≤ 8.3e-5 relative to range, cosine 1.0000000 |
| e2e logits | cosine 0.99998 |
| e2e sigmoid matte | cosine 0.99977 |
| foreground fraction | 0.0292, matching the oracle's 0.0292 exactly |
fp16 is the right dtype here, and bf16 is not. At the same gate bf16 fails (sigmoid cosine
0.9958, foreground fraction drifting to 0.0272) — its 8-bit mantissa is too coarse for this graph.
No bf16 variant is published for that reason.
The fp32 conversion is bit-exact against the source, so any residual difference above is Apple-GPU
fp32 accumulation, not conversion loss.
Usage (MLX-Swift)
1import MLXServeCore
2import MLXBiRefNet
3
4let engine = MLXServeEngine()
5let id = try await engine.register(Lucida.registration, configuration: .lucida())
6let resp = try await engine.run(MattingRequest(image: photo), package: id) as! MattingResponse
7// resp.matte — soft-alpha grayscale PNG at the input's dimensions
Measured on an M-series Max: ~0.38 s per 1024² matte, ~0.6 GB resident + ~14 GB peak transient.
Choosing between Lucida and BiRefNet
Lucida is not a drop-in replacement — the two lead on different material. On the author's own
203-image benchmark (MAE, lower is better):
| Lucida is better at | BiRefNet-HR is better at |
|---|
| camouflage 0.0270 vs 0.0752 · print/design 0.0235 vs 0.0544 · text & logos 0.0091 vs 0.0207 · illustration 0.0092 vs 0.0157 · transparency 0.0358 vs 0.0687 | hair 0.0048 vs 0.0093 · thin structures 0.0196 vs 0.0322 · complex edges 0.0385 vs 0.0484 |
Overall on that set: 0.0257 vs 0.0346. So route photographic portrait/hair work to BiRefNet and
non-photographic, hard-edge, text or illustration work here.
Those figures are the author's own — 203 self-selected images, 9 self-defined categories, MAE
only (which flatters soft-alpha smoothness), and the published table names "Lucida-v7" while the
source repo ships one unversioned checkpoint. Treat as directional, not decisive.
Also note the author's
colour-decontamination and edge-refinement passes are not part of the
checkpoint — they live in
their GitHub repo. This conversion
provides the network only, emitting raw sigmoid alpha.
License & provenance
MIT, following upstream's declaration for both the weights and the code.
One thing to be aware of before commercial use: the upstream card states the training data
"mix[es] MIT-licensed and research-only datasets" — P3M-10k, COD10K and DIS5K, alongside the
CC-BY-4.0
ToonOut set. That is a question about
the data the model was derived from rather than about the MIT grant we received, but it is worth your
own review rather than inheriting the label uncritically.
Credits
