You can deploy this model and start earning money today!

ArtusDev/
Alibaba-NLP_Tongyi-DeepResearch-30B-A3B-EXL3

Name: Alibaba-NLP_Tongyi-DeepResearch-30B-A3B-EXL3
Availability: InStock

Views

No views yet

ArtusDev/Alibaba-NLP_Tongyi-DeepResearch-30B-A3B-EXL3

EXL3 quants of Alibaba-NLP/Tongyi-DeepResearch-30B-A3B using exllamav3 for quantization.

You can download quants by targeting specific size using the Hugging Face CLI.

Click for download commands

1. Install huggingface-cli:

pip install -U "huggingface_hub[cli]"

2. Download a specific quant:

huggingface-cli download ArtusDev/Alibaba-NLP_Tongyi-DeepResearch-30B-A3B-EXL3 --revision "5.0bpw_H6" --local-dir ./

EXL3 quants can be run with any inference client that supports EXL3, such as TabbyAPI. Refer to documentation for set up instructions.

See EXL community hub for request guidelines.