Qwythos-9B-v2-MLX-4bit – AI Model by ahmedandaloes | AlphaNeural AI

You can deploy this model and start earning money today!

ahmedandaloes/
Qwythos-9B-v2-MLX-4bit

like

0

text-generation

empero-ai/Qwythos-9B-v2

Views

No views yet

Qwythos-9B-v2 — MLX 4-bit

4-bit quantized MLX build of empero-ai/Qwythos-9B-v2, for fast local inference on Apple Silicon.

Precision: 4-bit affine, group size 64.
Weights unchanged from source — format + precision conversion only.
Converted with mlx-lm.

Builds

Build
MLX-4bit
MLX-6bit
MLX-8bit

GGUF builds: empero-ai/Qwythos-9B-v2-GGUF.

Usage

pip install mlx-lm

python

from mlx_lm import load, generate
model, tok = load("ahmedandaloes/Qwythos-9B-v2-MLX-4bit")
p = tok.apply_chat_template([{"role":"user","content":"Name a common web vulnerability."}], add_generation_prompt=True)
print(generate(model, tok, prompt=p, max_tokens=200, verbose=True))

Attribution

Source: empero-ai/Qwythos-9B-v2. License per source (Apache-2.0 assumed; verify). MLX build for the Apple Silicon community. For authorized security work only.