You can deploy this model and start earning money today!

vincentzed-hf/
DeepSeek-V3.2-FP8

Name: DeepSeek-V3.2-FP8
Availability: InStock

Views

No views yet

DeepSeek V3.2 FP8 -> FP4 (E2E)

This is a minimal, V3.2-only end‑to‑end flow using the official V3.2 inference code (not V3.2-Exp). Each step says what it’s doing.

Set paths (only the vars actually used below)

bash

# NOTE: HF cache stores safetensors under snapshots/<hash>/, NOT at the model root!
# Find the correct path with: ls /root/.cache/huggingface/hub/models--deepseek-ai--DeepSeek-V3.2/snapshots/
export HF_FP8_CKPT="/root/.cache/huggingface/hub/models--deepseek-ai--DeepSeek-V3.2/snapshots/a7e62ac04ecb2c0a54d736dc46601c5606cf10a6"
export DS_CKPT="/sgl-workspace/sglang/Model-Optimizer/examples/deepseek/out/deepseek-v3.2-fp8"
export FP4_QUANT_PATH="/sgl-workspace/sglang/Model-Optimizer/examples/deepseek/out/deepseek-v3.2-fp4-calib"
export HF_FP4_PATH="/sgl-workspace/sglang/Model-Optimizer/examples/deepseek/out/deepseek-v3.2-fp4-hf"
export MP=4  # must match --nproc-per-node in PTQ step
export EXPERTS=256

Get the V3.2 inference repo (skip safetensors download)

bash

# work from the Model-Optimizer example directory so paths below match
cd /sgl-workspace/sglang/Model-Optimizer/examples/deepseek

# clone the official V3.2 inference repo without pulling LFS (avoids safetensors download)
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/deepseek-ai/DeepSeek-V3.2 && cd DeepSeek-V3.2

# we install FHT from src. see setup.py in FHT repo to see the more step / end var.
pip install git+https://github.com/Dao-AILab/fast-hadamard-transform.git
# install inference dependencies from the official repo
pip install -r inference/requirements.txt

Convert the HF checkpoint to DeepSeek format

bash

cd DeepSeek-V3.2/inference
python convert.py \
  --hf-ckpt-path $HF_FP8_CKPT \
  --save-path $DS_CKPT \
  --n-experts $EXPERTS \
  --model-parallel $MP

Run calibration (PTQ) for V3.2

bash

cd /sgl-workspace/sglang/Model-Optimizer/examples/deepseek
# --nproc-per-node MUST match --model-parallel used in convert.py (i.e. $MP)
CUDA_VISIBLE_DEVICES=4,5,6,7 torchrun --nproc-per-node $MP --master_port=12346 ptq.py \
  --model_path $DS_CKPT \
  --config DeepSeek-V3.2/inference/config_671B_v3.2.json \
  --quant_cfg NVFP4_DEFAULT_CFG \
  --output_path $FP4_QUANT_PATH

Quantize FP8 -> NVFP4 and assemble final HF checkpoint

bash

./quantize_fp8_to_nvfp4.sh \
  --amax_path $FP4_QUANT_PATH \
  --fp4_output_path $HF_FP4_PATH \
  --fp8_hf_path $HF_FP8_CKPT \
  --world_size $MP

(Optional) Run V3.2 inference to verify the converted checkpoint

bash

cd DeepSeek-V3.2/inference
export CONFIG=config_671B_v3.2.json
torchrun --nproc-per-node $MP generate.py --ckpt-path $DS_CKPT --config $CONFIG --interactive

vincentzed-hf/DeepSeek-V3.2-FP8

DeepSeek V3.2 FP8 -> FP4 (E2E)

Set paths (only the vars actually used below)

Get the V3.2 inference repo (skip safetensors download)

Convert the HF checkpoint to DeepSeek format

Run calibration (PTQ) for V3.2

Quantize FP8 -> NVFP4 and assemble final HF checkpoint

(Optional) Run V3.2 inference to verify the converted checkpoint

vincentzed-hf/
DeepSeek-V3.2-FP8