A lightweight neural codec that encodes audio at just 0.8 kbps - perfect for researchers and builders who need something that just works for training high quality text-to-speech models.
NeuCodec is a Finite Scalar Quantisation (FSQ) based 0.8kbps audio codec for speech tokenization.
It takes advantage of the following features:
NeuCodec is largely based on extending the work of
X-Codec2.0.
Use the code below to get started with the model.
1conda create -n neucodec python=3.10
2conda activate neucodec
3pip install neucodec
1import librosa
2import torch
3import torchaudio
4from torchaudio import transforms as T
5from neucodec import NeuCodec
6
7model = NeuCodec.from_pretrained("neuphonic/neucodec")
8model.eval().cuda()
9
10y, sr = torchaudio.load(librosa.ex("libri1"))
11if sr != 16_000:
12 y = T.Resample(sr, 16_000)(y)[None, ...] # (B, 1, T_16)
13
14with torch.no_grad():
15 fsq_codes = model.encode_code(y)
16 # fsq_codes = model.encode_code(librosa.ex("libri1")) # or directly pass your filepath!
17 print(f"Codes shape: {fsq_codes.shape}")
18 recon = model.decode_code(fsq_codes).cpu() # (B, 1, T_24)
19
20torchaudio.save("reconstructed.wav", recon[0, :, :], 24_000)
All publically available data was covered by either the CC-BY-4.0 or CC0 license.
