You can deploy this model and start earning money today!

Senat1/
dmx-llama-3.1-8b-m5

Source: NousResearch/Meta-Llama-3.1-8B-Instruct (FP16)
Format: DMX BFP M=7 (7 mantissa bits, block floating point)
File size: 5.54 GB (65% smaller than FP16)
Quality: Within GPU variance of FP16 (BF16-equivalent precision)

Name: dmx-llama-3.1-8b-m5
Availability: InStock

Views

No views yet

dmx-llama-3.1-8b-m5

pip install dmx-compress dmx-runtime

python

from dmx_runtime import from_dmx_compressed

model = from_dmx_compressed(
    "model.dmx",
    model_id="NousResearch/Meta-Llama-3.1-8B-Instruct"
)

Compressed with dmx-compress.