WMT22 COMET-KIWI DA FP16 (EOLE)
This is
Unbabel/wmt22-cometkiwi-da converted to
EOLE format.
Model weights were dtype-cast to fp16 during EOLE conversion. This repository contains a format and dtype conversion for use with EOLE's native transformer_encoder_scorer implementation.
This is not an upstream unbabel-comet checkpoint layout. Use it with EOLE.
Model Details
| |
|---|
| Original model | Unbabel/wmt22-cometkiwi-da |
| EOLE architecture | transformer_encoder_scorer |
| Weight dtype | fp16 |
| Scoring type | comet |
| Class identifier | unified_metric |
| EOLE scorer | EOLE-COMET-KIWI |
| Requires reference | No |
| Encoder | XLM-R large style encoder |
Usage
Requires an EOLE version with native COMET scorer support.
Validation Metric
1valid_metrics: ["EOLE-COMET-KIWI"]
2comet_model: eole-nlp/wmt22-cometkiwi-da-eole-fp16
3comet_batch_size: 64
Direct Scoring
COMET-KIWI is reference-free. Use --src for source sentences and --tgt for machine translations.
1eole predict \
2 --model_path eole-nlp/wmt22-cometkiwi-da-eole-fp16 \
3 --src /path/to/src.txt \
4 --tgt /path/to/mt.txt \
5 --output /path/to/scores.txt \
6 --with_score
By default, direct scoring writes one segment score per input line. To emit a
single aggregate system score, add --score_level system:
1eole predict \
2 --model_path eole-nlp/wmt22-cometkiwi-da-eole-fp16 \
3 --src /path/to/src.txt \
4 --tgt /path/to/mt.txt \
5 --output /path/to/system-score.txt \
6 --with_score \
7 --score_level system
score_level: system writes one numeric line containing the arithmetic mean of
the segment scores, similar to Unbabel COMET's --only_system mode but using
EOLE's score-file format.
Conversion
Converted with:
1eole convert COMET \
2 --model Unbabel/wmt22-cometkiwi-da \
3 --dtype fp16 \
4 --output wmt22-cometkiwi-da-eole-fp16
Parity
The fp32 EOLE conversion was parity-checked against the upstream Unbabel COMET runtime. On a 100-line WMT17 slice, native EOLE scoring matched the upstream baseline within floating point tolerance:
| Metric | Value |
|---|
| System score delta | 1.01e-08 |
| Sentence MAE | 1.28e-07 |
| Sentence max abs | 9.54e-07 |
The fp16 artifacts are intended to provide smaller EOLE model files and fp16 runtime behavior. Run the EOLE COMET parity harness against these fp16 repositories if exact fp16 deltas are required.
Original Model and Attribution
This model is a conversion of
Unbabel/wmt22-cometkiwi-da. The model weights were converted to EOLE format and dtype-cast to fp16.
This converted repository follows the upstream model license: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC-BY-NC-SA-4.0). Use is subject to the upstream non-commercial, attribution, and share-alike terms. Please refer to the original model card and license for training data, intended use, limitations, and citation details.
