This repository contains the optimized ONNX export of the jbochi/madlad400-3b-mt model,
optimized for reduced memory consumption following the NLLB optimization approach.
Model Description
Base Model: jbochi/madlad400-3b-mt
Optimization: Component separation for reduced RAM usage
Target: Mobile and edge deployment
Format: ONNX with separated components
Files Structure
Optimized Components (/model/)
madlad_encoder.onnx - Encoder component
madlad_decoder.onnx - Decoder component
madlad_decoder.onnx_data - Decoder weights data
tokenizer_config.json - Tokenizer configuration
special_tokens_map.json - Special tokens mapping
spiece.model - SentencePiece tokenizer model
inference_script.py - Python inference script
Original Models (/original_models/)
Complete original ONNX exports for reference
Optimization Benefits
Memory Reduction: Separated shared components to avoid duplication
Mobile Ready: Optimized for deployment on mobile devices
Modular: Components can be loaded independently as needed
Usage
python
1# Basic usage with the optimized models2from transformers import T5Tokenizer
3import onnxruntime as ort
45# Load tokenizer6tokenizer = T5Tokenizer.from_pretrained("manancode/madlad400-3b-mt-onnx-optimized", subfolder="model")78# Load ONNX models9encoder_session = ort.InferenceSession("model/madlad_encoder.onnx")10decoder_session = ort.InferenceSession("model/madlad_decoder.onnx")1112# For detailed inference, see inference_script.py
Translation Example
python
1# Input format: <2xx> text (where xx is target language code)2text ="<2pt> I love pizza!"# Translate to Portuguese3# Expected output: "Eu amo pizza!"
Language Codes
This model supports translation to 400+ languages. Use the format <2xx> where xx is the target language code:
<2pt> - Portuguese
<2es> - Spanish
<2fr> - French
<2de> - German
And many more...
Performance Notes
Original Model Size: ~3.3B parameters
Memory Optimization: Reduced RAM usage through component separation
Inference Speed: Optimized for faster generation with separated components
Technical Details
Optimization Approach
This optimization follows the same principles used for NLLB models:
Component Separation: Split encoder/decoder into separate files
Weight Deduplication: Avoid loading shared weights multiple times
Memory Efficiency: Load only required components during inference
