DeepSeek-V2-Lite NVFP4 Quantized
This repository contains an NVFP4-quantized version of DeepSeek-V2-Lite, prepared using TensorRT Model Optimizer.
The model has approximately 15.6B total parameters and is quantized to NVIDIA 4-bit floating-point precision (NVFP4).
This is my first quantized model.
Model Details
- Developed by: Krisakorn Chanthasang
- Model type: Large Language Model for text generation
- Languages: English, Chinese
- License: Apache 2.0
- Base model: deepseek-ai/DeepSeek-V2-Lite
- Quantization format: NVFP4
- Quantization tool: TensorRT Model Optimizer
Base Model Information
For details about the original model architecture, training data, and intended usage, see the official base model page:
Requirements
This model requires hardware and inference software with NVFP4 support.
Hardware Requirement
- NVIDIA Blackwell GPU or newer
- Tested/quantized using: NVIDIA RTX PRO 6000 Blackwell Workstation
How to Use
Use this model with an inference engine that supports NVFP4 quantized models.
Compatibility depends on your runtime, GPU architecture, and TensorRT/NVIDIA software stack.
Notes
This is a quantized derivative of DeepSeek-V2-Lite. Accuracy, throughput, and memory usage may differ from the original model depending on the inference engine and hardware used.
