You can deploy this model and start earning money today!

ibitec/
Qwen3-4B-Instruct-2507-Eagle3-MNN-SGLang

Name: Qwen3-4B-Instruct-2507-Eagle3-MNN-SGLang
Availability: InStock

Views

No views yet

Qwen3-4B-Instruct-2507-Eagle3-MNN

Introduction

This model is a 4-bit quantized version of the MNN model exported from Qwen3-4B-Instruct-2507-Eagle3 using llmexport.

Download

bash

# install huggingface
pip install huggingface

bash

# shell download
huggingface download --model 'taobao-mnn/Qwen3-4B-Instruct-2507-Eagle3-MNN' --local_dir 'path/to/dir'

python

# SDK download
from huggingface_hub import snapshot_download
model_dir = snapshot_download('taobao-mnn/Qwen3-4B-Instruct-2507-Eagle3-MNN')

bash

# git clone
git clone https://www.modelscope.cn/taobao-mnn/Qwen3-4B-Instruct-2507-Eagle3-MNN

Usage

bash

# clone MNN source
git clone https://github.com/alibaba/MNN.git

# compile
cd MNN
mkdir build && cd build
cmake .. -DMNN_LOW_MEMORY=true -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true
make -j

# run
./llm_demo /path/to/Qwen3-4B-Instruct-2507-Eagle3-MNN/config.json prompt.txt

Document

MNN-LLM

ibitec/Qwen3-4B-Instruct-2507-Eagle3-MNN-SGLang

Qwen3-4B-Instruct-2507-Eagle3-MNN

Introduction

Download

Usage

Document

ibitec/
Qwen3-4B-Instruct-2507-Eagle3-MNN-SGLang