You can deploy this model and start earning money today!

aoiandroid/
neutts-jp-150m-onnx-float32

Name: neutts-jp-150m-onnx-float32
Availability: InStock

Views

No views yet

neuTTS-JP-150m ONNX float32

This is the ONNX export of the neuTTS-JP-150m model, a Japanese text-to-speech model with voice cloning capabilities.

Model Details

Original Model: aoiandroid/neuTTS-JP-150m
Architecture: LlamaForCausalLM (151.8M parameters)
File Size: ~609 MB (1.75 MB ONNX + 607 MB data)
Format: ONNX (Opset 18)
Language: Japanese
Sampling Rate: 24 kHz

Repository Structure

aoiandroid/neutts-jp-150m-onnx/
├── neutts-jp-150m-original.onnx          # ONNX model structure (1.75 MB)
├── neutts-jp-150m-original.onnx.data      # ONNX model weights (607 MB)
└── README.md                              # Model card

File Descriptions:

neutts-jp-150m-original.onnx: ONNX model file containing the graph structure and metadata
neutts-jp-150m-original.onnx.data: External data file containing the model weights (large tensor data)
README.md: Model documentation and usage instructions

Usage

Installation

pip install onnxruntime transformers torchaudio neucodec soundfile

Python Example

python

import onnxruntime as ort
from transformers import AutoTokenizer
from neucodec import NeuCodec
import torch
import torchaudio
from torchaudio import transforms as T
import soundfile as sf

# Load ONNX model
session = ort.InferenceSession(
    "neutts-jp-150m-original.onnx",
    providers=['CPUExecutionProvider']
)

# Load tokenizer and codec
tokenizer = AutoTokenizer.from_pretrained(
    "aoiandroid/neuTTS-JP-150m",
    trust_remote_code=True
)
codec = NeuCodec.from_pretrained("neuphonic/neucodec")
codec.eval()

# Load and process reference audio
waveform, sr = torchaudio.load("reference_audio.wav")
if waveform.shape[0] > 1:
    waveform = waveform.mean(dim=0, keepdim=True)
if sr != 16_000:
    waveform = T.Resample(sr, 16_000)(waveform)
waveform = waveform.unsqueeze(0)

# Encode reference audio
with torch.inference_mode():
    ref_codes = codec.encode_code(waveform).flatten().tolist()

# Tokenize text
text = "こんにちは、これはテストです。"
text_ids = tokenizer.encode(text, add_special_tokens=False)
eos_id = int(tokenizer.eos_token_id)
input_ids = ref_codes + [eos_id] + text_ids + [eos_id]
input_ids = torch.tensor([input_ids], dtype=torch.long)

# Generate using ONNX model
input_ids_np = input_ids.numpy()
outputs = session.run(None, {'input_ids': input_ids_np})
logits = outputs[0]

# Get next token (greedy decoding)
next_token = int(logits[0, -1].argmax())
# (Repeat generation loop for full sequence)

# Decode audio
gen_ids = [generated_tokens]  # Your generated tokens
gen_ids_tensor = torch.tensor(gen_ids, dtype=torch.long).unsqueeze(0).unsqueeze(0)
with torch.inference_mode():
    audio_data = codec.decode_code(gen_ids_tensor).cpu()

# Save output
sf.write("output.wav", audio_data[0][0].cpu().numpy(), 24_000)

Model Architecture

This model is based on LlamaForCausalLM with the following configuration:

hidden_size: 512
num_hidden_layers: 12
num_attention_heads: 8
intermediate_size: 2048
vocab_size: 99111

Requirements

onnxruntime >= 1.25.0
transformers >= 4.29.0
torch >= 2.11.0
torchaudio >= 2.11.0
neucodec
soundfile

Original Model

aoiandroid/neuTTS-JP-150m

License

MIT License

Citation

If you use this model, please cite the original neuTTS-JP-150m model.

aoiandroid/neutts-jp-150m-onnx-float32