You can deploy this model and start earning money today!

soundsol/
helix-v0.7

Name: helix-v0.7
Availability: InStock

safetensors

audio

spatial-audio

first-order-ambisonics

Views

No views yet

Helix v0.7 - Mono to Spatial FOA

Helix is a neural spatial audio model that converts mono audio to First-Order Ambisonic (FOA) 4-channel format using text-based spatial positioning.

🎧 Try the demo on Spaces (Coming soon)

Model Description

Architecture: UNet1D with FiLM conditioning (2.4M parameters)
Input: Mono audio (24kHz) + spatial text description
Output: 4-channel FOA (W, X, Y, Z) at 24kHz
Training: 50 epochs on 2,490 commercial-safe paired samples
Validation Loss: 1.26 (88% improvement from initialization)

Capabilities

✅ Convert mono/stereo to spatial FOA ✅ Text-guided positioning (8 directions × 3 elevations × 3 distances) ✅ Real-time capable (CPU inference) ✅ 100% commercial-safe training data ✅ Compatible with VR/AR and spatial audio workflows

Usage

Quick Start

python

import torch
import soundfile as sf
from helix_model import HelixModel

# Load model
model = HelixModel.from_pretrained("your-username/helix-v0.7")

# Load mono audio
audio, sr = sf.read("your_audio.wav")

# Spatialize
foa_output = model.spatialize(
    audio,
    direction="left",
    elevation="level",
    distance="mid"
)

# Save FOA output (4 channels)
sf.write("output_foa.wav", foa_output.T, 24000)

Spatial Parameters

Direction (8 options):

front, front-left, left, back-left, back, back-right, right, front-right

Elevation (3 options):

down (-30°), level (0°), up (+30°)

Distance (3 options):

near (1m), mid (2.5m), far (5m)

Advanced Usage

python

# Batch processing
foa_outputs = model.spatialize_batch(
    audio_list,
    positions=[
        ("front", "level", "near"),
        ("left", "up", "far"),
        ("back", "down", "mid")
    ]
)

# Custom positions (azimuth, elevation in degrees)
foa = model.spatialize_custom(
    audio,
    azimuth=45.0,
    elevation=15.0,
    distance=2.0
)

FOA Output Format

The model outputs First-Order Ambisonics (SN3D normalization):

Channel 0 (W): Omnidirectional (pressure/mono sum)
Channel 1 (X): Front-back axis
Channel 2 (Y): Left-right axis
Channel 3 (Z): Up-down axis

Decoding FOA

python

# Decode to binaural for headphones
from helix_utils import foa_to_binaural

binaural = foa_to_binaural(foa_output)
sf.write("binaural.wav", binaural.T, 24000)

# Or use in spatial audio tools:
# - Reaper (with ambisonic plugins)
# - Pro Tools (with Dolby Atmos)
# - Unity/Unreal (native FOA support)

Training Data

Trained on 100% commercial-safe data:

Source	License	Samples	Type
FMA	CC BY 4.0	2,000	Music
Common Voice	CC0	490	Speech
pyroomacoustics	MIT	Generated	RIRs

Total: 2,490 paired mono→FOA examples

Performance

Inference Speed: ~0.15s for 4-second audio (CPU)
Model Size: 9.6 MB
Memory: ~500 MB RAM
Sample Rate: 24 kHz fixed
Max Duration: 30 seconds (longer audio will be chunked)

Limitations

⚠️ Current limitations:

Fixed 24kHz sample rate
Best for speech and music (limited effect sounds)
Trained on limited spatial diversity
No real-time streaming (processes full clips)

Citation

bibtex

@misc{helix2024,
  title={Helix: Text-Guided Mono to Spatial Audio},
  author={Your Name},
  year={2024},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/your-username/helix-v0.7}}
}

License

Model: CC BY 4.0 (commercial use allowed with attribution) Training Data: All commercial-safe (CC0, CC BY 4.0, MIT)

Model Card Contact

For questions or issues, please open an issue on GitHub or contact [your-email].

Example Applications

🎵 Music Production: Pan instruments in 3D space 🎙️ Podcasts: Position speakers spatially 🎮 Game Audio: Convert mono assets to spatial 🎬 Film: Quick spatial audio prototyping 🥽 VR/AR: Immersive audio experiences

Updates

v0.7 (Oct 2024):

Initial release
2.4M parameter UNet1D
Text-guided spatial positioning
100% commercial-safe training

Coming in v0.8:

20K training samples (8x more data)
Better speech quality (Common Voice)
Sound effects support (Freesound)
Real spatial recordings (TAU dataset)
Lower validation loss target: <1.0

Built with PyTorch Lightning | Gradio demo available

soundsol/helix-v0.7

Helix v0.7 - Mono to Spatial FOA

Model Description

Capabilities

Usage

Quick Start

Spatial Parameters

Advanced Usage

FOA Output Format

Decoding FOA

Training Data

Performance

Limitations

Citation

License

Model Card Contact

Example Applications

Updates

soundsol/
helix-v0.7