🚨 This model is 1bit quantized version of Cohere Labs Command A using QEP. You can find the unquantized version of Cohere Labs Command A here.
Model Summary
An optimized 1-bit quantized version of c4ai/command-a-03-2025 achieving 6.7x compression with enhanced performance through advanced quantization optimization techniques.
Key Features
Extreme Compression: 6.7× smaller (207GB → 30.2GB, -85%), runs even on a single GPU (30B on A100 80GB).
Enhanced Performance: Onebit quantization, enhanced by Fujitsu QEP & QQA.
Inference Speed Up: Faster inference via "Bitlinear computation".
Model Details
Base Model: c4ai/command-a-03-2025
Quantization Method: OneBit with Fujitsu QEP/QQA optimization
Quantization Bits: 1-bit for layers 0-61, FP16 for last 2 layers
The base architecture of this model is Command-A. To load and use the model, please use the CommandA model class:
Load model.safetensors, which contains the quantized weights.
Replace all layers except the last two with bitlinear implementations.
Keep the last two layers with non-quantized weights for optimal performance.
The model requires the included onebit_linear.py for proper quantized layer implementation. The weights contain parameters for each of the OneBit-specific a, S, and b components necessary for reconstruction.
Depending on the level of performance you wish to maintain, you may keep additional layers near the output unquantized.
Note: Direct loading support as an extension of the transformers package is planned for future releases.
Global and Block-wise Fine-tuning: Explore fine-tuning strategies, including block-wise methods, to further improve accuracy and robustness.
Complete Usage Examples: Provide detailed implementation guides for efficient single-GPU deployment.
Optimization Updates: Enhance performance with next-generation quantization techniques and improved reconstruction methods.
Currently, the quantization process preserves the last two layers in non-quantized weights to maintain output quality, while applying aggressive 1-bit quantization to the remaining layers. Future releases will integrate block-wise fine-tuning for additional performance gains.
Ethical Considerations
This model inherits the capabilities and limitations of the base Command A model. Please refer to the original model's documentation for ethical guidelines and potential biases.
Model Card Contact
For errors or additional questions about details in this model card, contact fj-qep@dl.jp.fujitsu.com
Terms of Use:
We hope that the release of this model will make community-based research efforts more accessible, by releasing the weights of a highly performant model to researchers all over the world. This model is governed by a CC-BY-NC, requires also adhering to Cohere Lab's Acceptable Use Policy
