You can deploy this model and start earning money today!

Thox-ai/
ThoxLLM-327M-v2-GGUF

Name: ThoxLLM-327M-v2-GGUF
Availability: InStock

Thox-ai/ThoxLLM-327M-v2

quantized

apache-2.0

endpoints_compatible

Views

No views yet

ThoxLLM-327M-v2-GGUF

Your AI. Your Data. Your Rules.

K-quant GGUF builds of ThoxLLM-327M-v2 for llama.cpp (K-quants only — legacy Q4_0/Q8_0 unsupported).

What this is

Quantized from Thox-ai/ThoxLLM-327M-v2 (Apache-2.0).
Only K-quants build cleanly: intermediate_size 3413 is not divisible by 32, so Q4_0/Q8_0 fail.
SHA256SUMS.txt included.

Intended use

On-device / edge text generation within the THOX stack. Not a safety-aligned public assistant unless deployed behind THOX guardrails.

Usage

llama.cpp

bash

huggingface-cli download Thox-ai/ThoxLLM-327M-v2-GGUF --include '*.gguf' --local-dir ./ThoxLLM-327M-v2-GGUF
llama-cli -m ./ThoxLLM-327M-v2-GGUF/model-Q2_K.gguf -p "Hello"

Thox-ai/ThoxLLM-327M-v2-GGUF

ThoxLLM-327M-v2-GGUF

What this is

Intended use

Usage

Links

Thox-ai/
ThoxLLM-327M-v2-GGUF