This is an improved version of
AfriCOMET-QE-STL (quality estimation single task) evaluation model: It receives a source sentence, and a translation, and returns a score that reflects the quality of the translation compared to the source.
Different from the original AfriCOMET-QE-STL, this QE model is based on an improved African enhanced encoder,
afro-xlmr-large-76L, which leads better performance on quality estimation of African-related machine translation, verified in WMT 2024 Metrics Shared Task.
1pip install --upgrade pip # ensures that pip is current
2pip install unbabel-comet
1from comet import download_model, load_from_checkpoint
2
3model_path = download_model("masakhane/africomet-qe-stl-1.1")
4model = load_from_checkpoint(model_path)
5data = [
6 {
7 "src": "Nadal sàkọọ́lẹ̀ ìforígbárí o ní àmì méje sóódo pẹ̀lú ilẹ̀ Canada.",
8 "mt": "Nadal's head to head record against the Canadian is 7–2.",
9 },
10 {
11 "src": "Laipe yi o padanu si Raoniki ni ere Sisi Brisbeni.",
12 "mt": "He recently lost against Raonic in the Brisbane Open.",
13 }
14]
15model_output = model.predict(data, batch_size=8, gpus=1)
16print (model_output)
Given a source sentence and a translation, the model outputs a single quality score between 0 and 1 where 1 represents a perfect translation.
