import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon-H1-Tiny-90M-Instruct"

model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype=torch.bfloat16,
  device_map="auto"
)

# Perform text generation

transformers serve tiiuae/Falcon-H1-Tiny-90M-Instruct

`llama.cpp`

You can find all GGUF files compatible with llama.cpp under our official collection - an example setup could be:

bash

brew install llama.cpp 
pip install huggingface_hub 
hf download tiiuae/Falcon-H1-Tiny-90M-Instruct-GGUF Falcon-H1-Tiny-90M-Instruct-Q8_0.gguf --local-dir ./ 
llama-cli ./Falcon-H1-Tiny-90M-Instruct-Q8_0.gguf -cnv 

`ollama`

ollama run hf.co/tiiuae/Falcon-H1-Tiny-90M-Instruct-GGUF:Q8_0

Apple `mlx`

mlx_lm.chat --model tiiuae/Falcon-H1-Tiny-90M-Instruct

vLLM

For vLLM, simply start a server by executing the command below:

bash

# pip install vllm>=0.9.0
vllm serve tiiuae/Falcon-H1-Tiny-90M-Instruct --tensor-parallel-size 2 --data-parallel-size 1

sglang

bash

python -m sglang.launch_server \
  --model ttiiuae/Falcon-H1-Tiny-90M-Instruct \
  --tensor-parallel-size 1 

Evaluation

For detailed evaluation of Falcon-H1-Tiny series, please refer to our technical blogpost

Useful links

View our release blogpost.
Feel free to join our discord server if you have any questions or to interact with our researchers and developers.

Citation

If the Falcon-H1-Tiny family of models were helpful to your work, feel free to give us a cite.

@misc{falcon_h1_tiny,
  title={Falcon-H1-Tiny: A series of extremely small, yet powerful language models redefining capabilities at small scale},
  author={Falcon-LLM Team},
  year={2026}, 
}

Fedir-Ilina/Falcon-H1-Tiny-90M-Instruct-safe

Table of Contents

TL;DR

Model Details

Model Description

Training details

Usage

Inference

🤗 transformers

llama.cpp

ollama

Apple mlx

vLLM

sglang

Evaluation

Useful links

Citation

Fedir-Ilina/
Falcon-H1-Tiny-90M-Instruct-safe

`llama.cpp`

`ollama`

Apple `mlx`