Views
No views yet
KBLab/megatron-bert-base-swedish-cased-600k.fill-mask/onnx/model.onnx for masked language modeling logitsfeature-extraction/onnx/model.onnx for last_hidden_state and cls_embeddingmodel.onnx - FP32 baselinemodel_fp16.onnx - FP16, recommendedmodel_int8.onnx - dynamic INT8model_uint8.onnx - dynamic UINT8model_q4.onnx - 4-bit MatMul quantizationmodel.onnx: 738.6 MBmodel_fp16.onnx: 369.4 MBmodel_int8.onnx: 185.6 MBmodel_uint8.onnx: 185.6 MBmodel_q4.onnx: 283.8 MBmodel.onnx: 539.0 MBmodel_fp16.onnx: 269.6 MBmodel_int8.onnx: 135.4 MBmodel_uint8.onnx: 135.4 MBmodel_q4.onnx: 252.4 MB0.000069last_hidden_state max diff: 0.000006cls_embedding cosine similarity: 1.0fp16: top-5 exact match, max diff 0.0131int8: top-5 drift after rank 2uint8: top-5 drift after rank 2q4: top-5 drift after rank 2fp16: CLS cosine 0.9999997q4: CLS cosine 0.7404int8: CLS cosine 0.9731uint8: CLS cosine 0.9717model_fp16.onnx by default.fp16; int8/uint8 preserve embeddings better than q4 on this model.int8 and uint8 for fill-mask if token ranking fidelity matters.1megatron-bert-base-swedish-cased-600k/
2├── fill-mask/
3│ ├── config.json
4│ ├── tokenizer.json
5│ ├── tokenizer_config.json
6│ ├── special_tokens_map.json
7│ └── onnx/
8│ ├── model.onnx
9│ ├── model_fp16.onnx
10│ ├── model_int8.onnx
11│ ├── model_uint8.onnx
12│ └── model_q4.onnx
13└── feature-extraction/
14 ├── config.json
15 ├── tokenizer.json
16 ├── tokenizer_config.json
17 ├── special_tokens_map.json
18 └── onnx/
19 ├── model.onnx
20 ├── model_fp16.onnx
21 ├── model_int8.onnx
22 ├── model_uint8.onnx
23 └── model_q4.onnxoptimum 2.1.0 task export does not support megatron-bert.torch.onnx.export wrappers using legacy TorchScript exporter (dynamo=False).cls_embedding = last_hidden_state[:, 0, :].MegatronBertModel from this MLM checkpoint would introduce randomly initialized pooler weights.