[Blog] [Code]

BLASER 2.0 is the new version of BLASER (Chen et al., 2023), a family of models for automatic evaluation of machine translation quality.

BLASER 2.0 is based on SONAR sentence embeddings and works with both speech and text modalities.

The actual model predicts a similarity score for the translated sentence based on the translation and the source sentence. This, it can be applied in settings where reference translations are missing or if their quality is questionable. In contrast, its sibling model, BLASER 2.0-referenced, requires also a reference translation.

Supervised BLASER models are trained to predict cross-lingual semantic similarity scores, XSTS (Licht et al., 2022), on a scale where 1 corresponds to completely unrelated sentences and 5 corresponds to fully semantically equivalent sentences. The models predictions, though, are unbounded and can occasionally surpass these limits.

See the SONAR github repo for the installation instructions.

BLASER 2.0 models accept 1024-dimensional SONAR sentence embeddings as inputs, and produce a single score as an output. The code below illustrates their usage with text embeddings:

With BLASER 2.0 models, SONAR text and speech embeddings can be used interchangeably.