You can deploy this model and start earning money today!

smorand/
hf-ibm-granite-speech

Name: hf-ibm-granite-speech
Availability: InStock

endpoints_compatible

Views

No views yet

IBM Granite Speech - Hugging Face Inference Endpoint

Custom handler for deploying IBM Granite Speech 3.3 8B as a speech-to-text API on Hugging Face Inference Endpoints.

Features

Speech-to-text transcription using IBM Granite Speech 3.3 8B
Supports multiple audio formats (WAV, MP3, FLAC, etc.)
Automatic audio preprocessing (mono conversion, resampling to 16kHz)
Configurable generation parameters
JSON response with transcription or detailed error info

Project Structure

hf-ibm-granite-speech/
├── handler.py          # Custom handler implementation
├── requirements.txt    # Python dependencies
├── README.md          # This file
└── CLAUDE.md          # AI documentation

Model Information

Model: ibm-granite/granite-speech-3.3-8b
Type: Speech-to-Text (Seq2Seq)
Architecture: Granite Speech with audio encoder
Requirements: 16kHz mono audio input (auto-converted by handler)

Deployment

Option 1: Deploy from Hugging Face Hub

Create a new model repository on Hugging Face Hub
Upload handler.py and requirements.txt to the repo
Go to Inference Endpoints
Create new endpoint:
- Select your model repository
- Choose GPU instance (recommended: A10G or A100)
- Set "Custom Handler" as the task
Deploy

Option 2: Deploy with Model Files

If you have local model weights:

Upload model files to your HF repo
Place handler.py and requirements.txt in the repo root
Deploy as above

API Usage

Request Format

Send audio as base64-encoded data:

bash

# Encode audio file to base64
AUDIO_BASE64=$(base64 -w 0 audio.wav)

curl -X POST https://your-endpoint.endpoints.huggingface.cloud \
  -H "Authorization: Bearer $HF_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"inputs\": \"$AUDIO_BASE64\",
    \"parameters\": {
      \"prompt\": \"can you transcribe the speech into a written format?\",
      \"max_new_tokens\": 200,
      \"num_beams\": 4
    }
  }"

Python Client Example

python

import base64
import requests

# Load and encode audio
with open("audio.wav", "rb") as f:
    audio_base64 = base64.b64encode(f.read()).decode("utf-8")

# Make request
response = requests.post(
    "https://your-endpoint.endpoints.huggingface.cloud",
    headers={"Authorization": f"Bearer {HF_TOKEN}"},
    json={
        "inputs": audio_base64,
        "parameters": {
            "prompt": "can you transcribe the speech into a written format?",
            "max_new_tokens": 200
        }
    }
)

result = response.json()
if "text" in result:
    print(f"Transcription: {result['text']}")
else:
    print(f"Error: {result['error']}")

Response Format

Success:

json

{
  "text": "THIS IS THE TRANSCRIBED TEXT FROM THE AUDIO."
}

Error:

json

{
  "error": "ErrorType: Error message\nFull stack trace..."
}

Parameters

Parameter	Type	Default	Description
`inputs`	string	required	Base64-encoded audio data
`prompt`	string	"can you transcribe..."	Instruction prompt for the model
`system_prompt`	string	(default IBM prompt)	System prompt for the model
`max_new_tokens`	int	200	Maximum tokens to generate
`num_beams`	int	4	Number of beams for beam search
`do_sample`	bool	false	Whether to use sampling
`temperature`	float	1.0	Sampling temperature
`top_p`	float	1.0	Nucleus sampling parameter
`repetition_penalty`	float	3.0	Penalty for repeated tokens
`length_penalty`	float	1.0	Length penalty for beam search

Audio Requirements

The handler automatically preprocesses audio:

Supported formats: WAV, MP3, FLAC, OGG, and other formats supported by torchaudio
Auto-conversion: Stereo to mono, resampling to 16kHz
Recommended: 16kHz mono WAV for best results

Hardware Requirements

Minimum: A10G (24GB VRAM)
Recommended: A100 (40GB+ VRAM) for faster inference
Memory: ~16GB VRAM for the 8B model

Customization

Use Different Prompt

You can customize the transcription prompt:

json

{
  "inputs": "<base64-audio>",
  "parameters": {
    "prompt": "Please provide a detailed transcription with punctuation."
  }
}

Change Default Model

Set the MODEL_ID environment variable in your endpoint configuration:

MODEL_ID=your-org/your-fine-tuned-granite-speech

Troubleshooting

Out of Memory

Use a larger GPU instance (A100 instead of A10G)
Reduce max_new_tokens
Process shorter audio clips

Audio Format Errors

Ensure audio is properly base64 encoded
Check that the audio file is not corrupted
Try converting to 16kHz mono WAV before encoding

Slow Inference

Use A100 for faster inference
Reduce num_beams to 2
Use shorter audio clips

License

The handler code is provided as-is. IBM Granite Speech is subject to the Apache 2.0 License.

smorand/hf-ibm-granite-speech

IBM Granite Speech - Hugging Face Inference Endpoint

Features

Project Structure

Model Information

Deployment

Option 1: Deploy from Hugging Face Hub

Option 2: Deploy with Model Files

API Usage

Request Format

Python Client Example

Response Format

Parameters

Audio Requirements

Hardware Requirements

Customization

Use Different Prompt

Change Default Model

Troubleshooting

Out of Memory

Audio Format Errors

Slow Inference

License

smorand/
hf-ibm-granite-speech