[Blog] [Code]

BLASER 2.0 is the new version of BLASER (Chen et al., 2023), a family of models for automatic evaluation of machine translation quality.

BLASER 2.0 is based on SONAR sentence embeddings and works with both speech and text modalities.

The actual model predicts a similarity score for the translated sentence based on the the translation, the source sentence, and the reference translation. Its sibling model, BLASER 2.0 QE, does not use references.

Supervised BLASER models are trained to predict cross-lingual semantic similarity scores, XSTS (Licht et al., 2022), on a scale where 1 corresponds to completely unrelated sentences and 5 corresponds to fully semantically equivalent sentences. The models predictions, though, are unbounded and can occasionally surpass these limits.

See the SONAR github repo for the installation instructions.

BLASER 2.0 models accept 1024-dimensional SONAR sentence embeddings as inputs, and produce a single score as an output. The code below illustrates their usage with text embeddings:

With BLASER 2.0 models, SONAR text and speech embeddings can be used interchangeably.