SSA-COMET-STL, a robust, automatic metric for MTE, built based on SSA-MTE: It receives a triplet with (source sentence, translation, reference translation), and returns a score that reflects the quality of the translation.
This model is based on an improved African enhanced encoder,
afro-xlmr-large-76L.
1pip install --upgrade pip # ensures that pip is current
2pip install unbabel-comet
1from comet import download_model, load_from_checkpoint
2model_path = download_model("McGill-NLP/ssa-comet-stl")
3model = load_from_checkpoint(model_path)
4data = [
5 {
6 "src": "Nadal sàkọọ́lẹ̀ ìforígbárí o ní àmì méje sóódo pẹ̀lú ilẹ̀ Canada.",
7 "mt": "Nadal's head to head record against the Canadian is 7–2.",
8 "ref": "Nadal scored seven unanswered points against Canada."
9 },
10 {
11 "src": "Laipe yi o padanu si Raoniki ni ere Sisi Brisbeni.",
12 "mt": "He recently lost against Raonic in the Brisbane Open.",
13 "ref": "He recently lost to Raoniki in the game Sisi Brisbeni."
14 }
15]
16model_output = model.predict(data, batch_size=8, gpus=1)
17print (model_output)
Given a triplet with (source sentence, translation, reference translation), it outputs a single score between 0 and 1, where 1 represents a perfect translation.
