Local Voice Quality Enhancement — compact neural models for acoustic echo
cancellation (AEC), noise suppression (NS), and dereverberation of 16 kHz
speech, running on commodity CPUs in real time. Causal and streaming
(256-sample hop, 16 ms latency).
Joint models (v1.2 / v1.3) clean echo, noise, and reverb in one pass.
v1.3 is wider and filters noise better; v1.2 is ~1/4 the per-hop cost.
v1.4-AEC removes only the far-end echo and passes voice, room, and
background through unchanged. It's a classical adaptive filter followed by a
small neural mask. The 2.7K build is that filter alone — cheaper and
gentler, but it can't remove heavily reverberant echo the way the mask can.
Every model needs a far-end reference signal (a loopback of what your
speakers play) in addition to the mic.
bf16 GGUFs are ~12 % smaller with identical quality and speed; pick f32
unless download size matters.
Compact line — GTCRN-AEC (for lower-power CPUs)
A separate, much smaller second line of models for lower-power CPUs: a
~49 K-parameter GTCRN-AEC network — a distinct architecture based on
GTCRN (Rong et al., ICASSP 2024) —
paired with the project's DSP echo-cancellation front-end. The GGUFs are
self-contained, so they run with the same single command as every other model.
Two variants share the architecture:
Model
Does
Params
localvqe-pi-v1-49k
AEC + NS + dereverb (full enhance)
49 K
localvqe-pi-aec-v1-49k
echo only — keeps noise + room
49 K
Whole-clip real-time factor on the real ggml graph, benchmarked on a Raspberry
Pi 5 (one example of a low-power target; test_gtcrn --bench, Cortex-A76,
Ubuntu 24.04), parity-verified to the PyTorch reference within ~1e-6 on-device.
RTF is identical for both variants:
Threads
8 s clip
RTF
RT factor
1
388 ms
0.048
~21×
2
219 ms
0.027
~37×
4
163 ms
0.020
~49×
That is ~0.78 ms per 16 ms hop single-threaded. Runs on any CPU; for single-board
ARM, cross-compile for aarch64 with ggml/docker/Dockerfile.arm64 (docker buildx
qemu). f16/q8 quantizations are published only if/when released.
Files in this repository
File
Size
Model
localvqe-v1.4-aec-200K-f32.gguf
3 MB
v1.4-AEC (echo only)
localvqe-v1.4-aec-200K-bf16.gguf
2.6 MB
v1.4-AEC, conv weights in BF16
localvqe-v1.4-aec-2.7K-f32.gguf
17 KB
v1.4-AEC front-end only (adaptive filter, no mask)
localvqe-pi-v1-49k-f32.gguf
2.3 MB
Compact line — GTCRN-AEC full enhance (echo + NS + dereverb)
localvqe-pi-aec-v1-49k-f32.gguf
2.3 MB
Compact line — GTCRN-AEC echo-only (keeps noise + room)
v1.4-AEC is GGUF-only (no .pt). GGUF integrity is checked at load time against
a built-in SHA256 allowlist in the engine.
Performance
Full 800-clip eval on the
ICASSP 2022 AEC Challenge blind test set
(real recordings). AECMOS echo / deg are 1–5 (higher = more echo removed /
cleaner speech); blind ERLE is 10·log10(E[mic²]/E[enh²]), only meaningful on
far-end-only clips. Unprocessed-mic echo MOS is 2.67 / 2.56 / 1.90 / 2.13 / 5.00
across the five scenarios.
v1.4-AEC — keeps background noise and room by design, so its ERLE and
far-end DNSMOS are intentionally lower than the joint models (it isn't deleting
the ambience):
Scenario
n
echo ↑
deg ↑
ERLE ↑
OVRL
doubletalk
115
4.20
2.45
—
2.59
doubletalk-with-movement
185
4.19
2.45
—
2.55
farend-singletalk
107
3.80
4.99
14.6 dB
1.37
farend-singletalk-with-movement
193
3.86
4.95
11.1 dB
1.31
nearend-singletalk
200
4.99
3.99
—
3.08
v1.4-AEC 2.7K (front-end only) — matches or beats the full model's
perceptual far-end echo at 1/74 the parameters; the mask's extra work shows up
as higher ERLE above, not higher echo MOS:
Scenario
n
echo ↑
deg ↑
ERLE ↑
OVRL
doubletalk
115
4.00
2.79
—
2.46
doubletalk-with-movement
185
3.90
2.92
—
2.42
farend-singletalk
107
4.06
5.00
6.5 dB
1.24
farend-singletalk-with-movement
193
4.05
4.97
3.9 dB
1.22
nearend-singletalk
200
4.98
3.77
—
3.03
v1.3 (joint) and v1.2 (joint) — these also delete the background, so
their far-end ERLE is much higher and not comparable to v1.4-AEC's:
Scenario
n
v1.3 echo / deg / ERLE / OVRL
v1.2 echo / deg / ERLE / OVRL
doubletalk
115
4.73 / 2.62 / 8.5 dB / 2.89
4.72 / 2.37 / 8.4 dB / 2.83
doubletalk-with-movement
185
4.67 / 2.43 / 8.3 dB / 2.85
4.65 / 2.30 / 8.1 dB / 2.79
farend-singletalk
107
3.69 / 4.83 / 50.9 dB / 1.94
3.78 / 4.91 / 45.7 dB / 1.80
farend-singletalk-with-movement
193
3.88 / 4.98 / 49.9 dB / 1.96
4.12 / 4.96 / 40.6 dB / 1.75
nearend-singletalk
200
5.00 / 4.18 / 2.4 dB / 3.17
5.00 / 4.16 / 2.1 dB / 3.17
Latency
Per-hop p50 / RT factor on a Ryzen 9 7900 (Zen4). 16 kHz, 256-sample hop.
Model
1 thread
4 threads
dGPU (RTX 5070 Ti, Vulkan)
v1.4-AEC (203 K)
1.29 ms · 12.2×
0.83 ms · 18.6×
run on CPU¹
v1.4-AEC 2.7K
0.36 ms · 44× (single-threaded)
—
run on CPU¹
v1.3 (4.8 M)
9.73 ms · 1.58×
3.21 ms · 4.97×
2.57 ms · 6.07×
v1.2 (1.3 M)
4.28 ms · 3.72×
1.65 ms · 8.90×
1.96 ms · 7.85×
¹ v1.4-AEC's adaptive front-end always runs on CPU and the neural stage is too
small for GPU offload to pay off. Four threads is the sweet spot on Zen4 for all
models; the library defaults to min(4, available CPUs).
Memory (CPU)
Working set the model adds on top of the ~7 MiB binary baseline:
Model
Post-load delta
Peak RSS
v1.3 (4.8 M)
+24.4 MiB
34.1 MiB
v1.2 (1.3 M)
+10.0 MiB
19.6 MiB
v1.4-AEC (203 K)
+6.7 MiB
17.0 MiB
Running inference
Download a GGUF (web UI, huggingface-cli, or hf_hub_download) and run the
GGML CLI — same command for every model, just swap the file:
16 kHz mono PCM for both the mic and the far-end reference. Building the engine,
the C API (liblocalvqe.so), and the OBS Studio plugin are documented in the
GitHub repository.
PyTorch reference
localvqe-v1.3-4.8M.pt and localvqe-v1.2-1.3M.pt are the checkpoints used to
produce the GGUF exports — for verification, ablation, and research, not
end-user inference (use the GGML build). The model definition lives under
pytorch/ in the GitHub repo.
Citing
Cite the repository via CITATION.cff at
https://github.com/localai-org/LocalVQE (GitHub's "Cite this repository"
button produces APA / BibTeX), and the upstream DeepVQE paper:
bibtex
1@inproceedings{indenbom2023deepvqe,
2 title = {DeepVQE: Real Time Deep Voice Quality Enhancement for Joint
3 Acoustic Echo Cancellation, Noise Suppression and Dereverberation},
4 author = {Indenbom, Evgenii and Beltr{\'a}n, Nicolae-C{\u{a}}t{\u{a}}lin
5 and Chernov, Mykola and Aichner, Robert},
6 booktitle = {Interspeech}, year = {2023},
7 doi = {10.21437/Interspeech.2023-2176}
8}
The compact GTCRN-AEC line is based on GTCRN — please also cite:
bibtex
1@inproceedings{rong2024gtcrn,
2 title = {GTCRN: A Speech Enhancement Model Requiring Ultralow
3 Computational Resources},
4 author = {Rong, Xiaobin and Sun, Tianchi and Zhang, Xu and Hu, Yuxiang
5 and Zhu, Changbao and Lu, Jing},
6 booktitle = {ICASSP 2024 - 2024 IEEE International Conference on Acoustics,
7 Speech and Signal Processing (ICASSP)},
8 pages = {971--975}, year = {2024},
9 doi = {10.1109/ICASSP48485.2024.10448310}
10}
Training data was filtered by DNSMOS, which can misclassify distressed speech
(screaming, crying) as noise. LocalVQE may attenuate such signals and must not
be relied upon for emergency or safety-critical applications.
