Method: Optimum dynamic INT8 quantization
Source: BAAI/bge-reranker-v2-m3 (multilingual cross-encoder, XLM-RoBERTa-large, 568M params)
Size: ~570 MB (vs ~2.3 GB for the FP32 source)

Usage (Transformers.js)

import { AutoTokenizer, AutoModel } from '@huggingface/transformers';

const model = await AutoModel.from_pretrained(
  'tss-deposium/bge-reranker-v2-m3-onnx-int8',
  { dtype: 'int8' }  // or 'q8' for legacy file name
);

Notes

Cross-encoder: tokenize (query, doc) together, score from logits.
Multilingual (100+ langs); handles cross-language reranking better than bge-reranker-base.
For the smaller variant (279 MB, faster but weaker cross-lang), see Xenova/bge-reranker-base.