Unofficial GGUF conversion of eldavoo/emma-5, archived from the original Egomnia Emma-5 release.
The conversion was validated by reconstructing the ONNX attention graph in PyTorch and comparing logits against the original ONNX Runtime output before exporting to GGUF.
The original ONNX model remains the reference runtime. This GGUF is intended for raw llama.cpp completion use, not chat-template frontends.
Files
File
Description
emma-5-f16.gguf
FP16 GGUF, ~1.2 GB
Conversion Details
The original model is distributed as ONNX only. Direct conversion to GGUF required matching the exported attention graph exactly.
Pipeline summary:
ONNX weights and rotary constants were loaded.
The exported attention graph was matched in PyTorch for validation.
Logit parity was checked against ONNX Runtime.
Q/K rows were permuted from split-half RoPE layout to llama.cpp interleaved layout.
The model was exported as FP16 GGUF with rms_norm_eps=1e-6.
Validation result:
max absolute logit error: ~4e-5
Architecture
Parameter
Value
Parameters
~550M
Layers
28
Hidden size
1280
Q heads
16
KV heads
4
Head dim
80
GQA ratio
4:1
FFN
SwiGLU
Context
2048
Vocab
50000
RoPE theta
10000
Embeddings
Tied
Prompt Format
Use a single-line Italian instruction format:
### Istruzione: Qual e la capitale d Italia? ### Risposta:
Use raw completion mode only.
Do not use ChatML, Llama 3 chat templates, OpenAI-style messages, or frontend "conversation" wrappers. In particular, prompts containing markers like <|im_start|> or <|im_end|> are not supported and may cause repeated marker output.
Avoid multiline templates in llama.cpp. The original ONNX SentencePiece path normalizes whitespace differently, while llama.cpp may preserve newline tokens.
For the most faithful behavior, compare against or use the original ONNX Runtime model.
Example
bash
1./llama-cli \2 -m emma-5-f16.gguf \3 --no-conversation \4 -p "### Istruzione: Qual e la capitale d Italia? ### Risposta:"\5 -n 64\6 --temp 0.7\7 --repeat-penalty 1.2
Expected style:
La capitale d'Italia è Roma...
Quantization
This repo provides the FP16 GGUF. Other quantizations can be generated with llama.cpp:
