We present BLOOMZ & mT0, a family of models capable of following human instructions in dozens of languages zero-shot. We finetune BLOOM & mT5 pretrained multilingual language models on our crosslingual task mixture (xP3) and find the resulting models capable of crosslingual generalization to unseen tasks & languages.
pip install llm-rs1from llm_rs import AutoModel
2
3#Load the model, define any model you like from the list above as the `model_file`
4model = AutoModel.from_pretrained("rustformers/bloomz-ggml",model_file="bloomz-3b-q4_0-ggjt.bin")
5
6#Generate
7print(model.generate("The meaning of life is"))git clone --recurse-submodules https://github.com/rustformers/llm.git
cd llm
cargo build --releasecargo run --release -- bloom infer -m path/to/model.bin -p "Tell me how cool the Rust programming language is:"