Qwen3-4B-FlagOS-cambricon provides an all-in-one deployment solution, enabling execution of Qwen3-4B on cambricon GPUs. As the first-generation release for the cambricon-MLU590, this package delivers two key features:
We use FlagScale as the serving engine to improve the portability of distributed inference.
FlagScale is an end-to-end framework for large models across multiple chips, maximizing computational resource efficiency while ensuring model effectiveness. It ensures both ease of use and high performance for users when deploying models across different chip architectures:
We validate the execution of Qwen3-4B model with a Triton-based operator library as a PyTorch alternative.
We use a variety of Triton-implemented operation kernels to run the Qwen3-4B model. These kernels come from two main sources:
-
Most Triton kernels are provided by FlagGems (
https://github.com/FlagOpen/FlagGems). You can enable FlagGems kernels by setting the environment variable USE_FLAGGEMS.
-
Also included are Triton kernels from vLLM, such as fused MoE.
As requested by Cambricon, the Docker image and model files must be requested via email. Please contact
ecosystem@cambricon.com and include your organization name, contact person, contact information, equipment source, and specific requirements.
1pip install modelscope
2modelscope download --model Qwen/Qwen3-4B --local_dir /share/Qwen3-4B
3
1docker run -d --name flagos -e DISPLAY=$DISPLAY --net=host --pid=host --ipc=host -v /tmp/.X11-unix:/tmp/.X11-unix --privileged -v /share/project/shihonghao/host02:/share -v /home:/home -v /mnt/:/mnt/ -v /data/:/data/ -v /opt/data/:/opt/data/ -v /usr/bin/cnmon:/usr/bin/cnmon <IMAGE> sleep infinity
2
3docker exec -it flagos bash
