import torch
import torch_neuronx
from transformers import AutoProcessor

processor = AutoProcessor.from_pretrained("Qwen/Qwen3-ASR-1.7B")
encoder = torch.jit.load("encoder.pt")
decoder = torch.jit.load("decoder.pt")

# Always pad mel to exactly 3000 frames
mel = processor(audio, return_tensors="pt").input_features  # [1, 128, T]
if mel.shape[-1] < 3000:
    mel = torch.nn.functional.pad(mel, (0, 3000 - mel.shape[-1]))
feature_lens = torch.tensor([3000])  # always 3000, not actual length

encoder_out = encoder(mel, feature_lens)
# ... autoregressive decode with decoder

Key Tracing Notes

Fixed shape: Encoder always receives 3000 frames (30s). Shorter audio must be zero-padded.
feature_lens: Must always be torch.tensor([3000]) — actual length causes shape mismatch during tracing.
Core isolation: Run with NEURON_RT_VISIBLE_CORES=1,NEURON_RT_NUM_CORES=1 to use Core 1 alongside an LLM on Core 0.

Dual-Model Deployment

Pair with aqidd/qwen3-8b-int8-inf2 (LLM on Core 0) for a single inf2.xlarge serving both:

bash

# LLM on Core 0
NEURON_RT_VISIBLE_CORES=0 NEURON_RT_NUM_CORES=1 vllm serve Qwen/Qwen3-8B ...

# ASR on Core 1
NEURON_RT_VISIBLE_CORES=1 NEURON_RT_NUM_CORES=1 python3 asr_server.py

Related Models

aqidd/qwen3-8b-int8-inf2 — LLM companion (Core 0)

aqidd/qwen3-asr-1.7b-inf2

Qwen3-ASR 1.7B for AWS Inferentia2

Model Details

Artifacts

Usage

Key Tracing Notes

Dual-Model Deployment

Related Models

aqidd/
qwen3-asr-1.7b-inf2