We provide inference code for our BlackMamba model in our github repository:
https://github.com/Zyphra/BlackMambaBlackMamba is an novel architecture which combines state-space models (SSMs) with mixture of experts (MoE). It uses
Mamba as its SSM block and
switch transformer as its MoE block base. BlackMamba is extremely low latency for generation and inference, providing significant speedups over all of classical transformers, MoEs, and Mamba SSM models. Additionally, due to its SSM sequence mixer, BlackMamba retains linear compuational complexity in the sequence length.
