Distill-NeuCodec is a version of NeuCodec with a compatible, distilled encoder.
The distilled encoder is 10x smaller in parameter count and uses ~7.5x less MACs at inference time.
Our work is largely based on extending the work of
X-Codec2.0 and
SQCodec.
Use the code below to get started with the model.
1conda create -n neucodec python=3.10
2conda activate neucodec
3pip install neucodec
1import librosa
2import torch
3import torchaudio
4from torchaudio import transforms as T
5from neucodec import DistillNeuCodec
6
7model = DistillNeuCodec.from_pretrained("neuphonic/distill-neucodec")
8model.eval().cuda()
9
10y, sr = torchaudio.load(librosa.ex("libri1"))
11if sr != 16_000:
12 y = T.Resample(sr, 16_000)(y)[None, ...] # (B, 1, T_16)
13
14with torch.no_grad():
15 fsq_codes = model.encode_code(y)
16 # fsq_codes = model.encode_code(librosa.ex("libri1")) # or directly pass your filepath!
17 print(f"Codes shape: {fsq_codes.shape}")
18 recon = model.decode_code(fsq_codes).cpu() # (B, 1, T_24)
19
20torchaudio.save("reconstructed.wav", recon[0, :, :], 24_000)
The model was trained using the same data as the full model, with an additional distillation loss (MSE between distilled and original encoder ouputs).
