Metric	Value
In-memory footprint	~350 GiB (estimated, 1:1 with disk)
Size on disk	376.8 GB (86 shards)
Average bits per weight	3.95
Group size	64
Framework	MLX (Apple Silicon)
Source	deepseek-ai/DeepSeek-V3.2 (FP8 E4M3, 689.5 GB)
Compression vs source	46% smaller (689.5 GB to 376.8 GB)

Benchmarks

Benchmarks are pending — this model requires a 384 GB+ Mac to load.

Recommended inference settings

python

sampler_params = {
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 40,
    "repetition_penalty": 1.1,
    "max_tokens": 8192,
}

Chat template — thinking mode

DeepSeek-V3.2 uses a <think>...</think> reasoning block. The model begins its response inside a thinking block. Strip everything up to and including the first </think> to get the final answer:

python

def strip_thinking(text: str) -> str:
    if "</think>" in text:
        return text.split("</think>", 1)[1].strip()
    return text.strip()

Note: DeepSeek-V3.2 does not ship a Jinja chat template. For advanced chat formatting (tool calling, multi-turn), refer to the encoding scripts in the source repo.

Usage

python

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler, make_logits_processors

model, tokenizer = load("baa-ai/DeepSeek-V3.2-RAM-350GB-MLX")

sampler = make_sampler(temp=1.0, top_p=0.95, top_k=40)
logits_processors = make_logits_processors(repetition_penalty=1.1)

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Write a Python function that reverses a string."}],
    tokenize=False,
    add_generation_prompt=True,
)

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=8192,
    sampler=sampler,
    logits_processors=logits_processors,
)

if "</think>" in response:
    response = response.split("</think>", 1)[1].strip()
print(response)

Hardware

Apple Silicon Mac with 384 GB+ unified memory required.
Fits comfortably on a 512 GB Mac Studio Ultra or Mac Pro.
Does not fit on 192 GB machines without heavy swap.

Variants

Variant	Size	Link
350 GB	376.8 GB	baa-ai/DeepSeek-V3.2-RAM-350GB-MLX

Black Sheep AI Products

Shepherd — Private AI deployment platform that shrinks frontier models by 50-60% through RAM compression, enabling enterprises to run sophisticated AI on single GPU instances or Apple Silicon hardware. Deploy in your VPC with zero data leaving your infrastructure. Includes CI/CD pipeline integration, fleet deployment across Apple Silicon clusters, air-gapped and sovereign deployment support, and multi-format export (MLX, GGUF). Annual cloud costs from ~$2,700 — or run on a Mac Studio for electricity only.

Watchman — Capability audit and governance platform for compressed AI models. Know exactly what your quantized model can do before it goes live. Watchman predicts which capabilities survive compression in minutes — replacing weeks of benchmarking. Includes compliance-ready reporting for regulated industries, quality valley warnings for counterproductive memory allocations, instant regression diagnosis tracing issues to specific tensors, and 22 adversarial security probes scanning for injection, leakage, hallucination, and code vulnerabilities.

Learn more at baa.ai — Sovereign AI.

License

MIT — inherited from deepseek-ai/DeepSeek-V3.2.

Quantized by baa.ai