nos-mt-gl-en-onnx

Contents

Use

Preprocessing

• The source text gets no end-of-sentence token. The CT2 binary ships add_source_eos: false, so the encoder never saw </s>.
• Source and target have separate vocabularies. Pegasus has one, so the single table is [target vocabulary | source vocabulary] and every encoder input id is offset by 29648 (the target vocabulary size). The source half is suppressed at decode time with final_logits_bias = -1e9, so the decoder can never emit a source-side id.
• Words must be Moses-tokenized and BPE-applied first. The output is joined, @@ is removed, and the result is Moses-detokenized.
• Decoding starts from <s>.
• Both vocabularies are frequency-filtered, so rare words come back as <unk>. CTranslate2 prints <unk> as well. The upstream translate.py hides it with replace_unknowns=True, which copies the aligned source word; that needs attention alignments, which generate does not expose. nos_tokenizer.py therefore keeps <unk> in the output, so what you read is what the model produced.
• Feed one sentence at a time. The model has no document context and no language tag.

Parity with the original

Sample output

How the reconstruction works

Why Pegasus

1. Pre-norm blocks with a final encoder and decoder layer norm, and no layernorm_embedding. That rules out BART, mBART and PLBart, whose layernorm_embedding cannot be neutralised — a LayerNorm with weight 1 and bias 0 still normalises. It also rules out Marian, which is post-norm.
2. An output bias (decoder/projection/bias). Neither Marian nor M2M100 has one. Pegasus does, as final_logits_bias.

Traps

• Pegasus refuses to save its position table. embed_positions.weight is in _keys_to_ignore_on_save and is rebuilt on load with 10000^(2i/dim). OpenNMT-py interleaves sin and cos instead. This binary does store the real table, so the converter writes it in, clears _keys_to_ignore_on_save, and reloads the checkpoint to assert the table survived. Skipping this produces a model that runs and translates plausibly but wrongly.
• CTranslate2 fuses self-attention Q, K and V into one linear_0 of shape (3d, d) in that order. Cross-attention splits differently: linear_0 is Q alone, linear_1 is [K; V] fused, linear_2 is the output projection.
• These models were trained with add_qkvbias=False, so the attention and feed-forward projections carry no bias. Zeros are written where HuggingFace insists on one.
• Weights are stored (out, in), the layout torch.nn.Linear uses, so nothing is transposed. gamma and beta are the layer-norm weight and bias.
• int8 quantization is restricted to MatMul with /lm_head/MatMul excluded. Quantizing every operator destroys a 512-dimension NMT decoder.

Attribution