File	Bits/param	Use case
`deepseek-v2-lite-chat-Q4_K_M.gguf`	~4.5	Smallest, modest VRAM
`deepseek-v2-lite-chat-Q5_K_M.gguf`	~5.5	Sweet spot quality / size
`deepseek-v2-lite-chat-Q8_0.gguf`	~8	Near-lossless

Usage

Load with any GGUF-compatible runtime (llama.cpp, llama-cpp-python, LM Studio, Ollama, etc.):

./llama-cli -m <file>-Q4_K_M.gguf -p "Hello"

Notes

MoE model: 15.7B total parameters, 2.4B activated per token. Uses MLA (Multi-head Latent Attention) which compresses the key-value cache into a latent vector for efficient long-context inference.

The original model is released under DeepSeek's custom license; see the upstream repo for terms.

ggfox00000/deepseek-v2-lite-chat-gguf

DeepSeek-V2-Lite-Chat · GGUF

Source

Quantization

Files

Usage

Notes

ggfox00000/
deepseek-v2-lite-chat-gguf