# pip install transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "anothy1/SmolLM2-MoE-214M-A135M"
device = "cuda" # for GPU usage or "cpu" for CPU usage
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint, trust_remote_code=True).to(device)
inputs = tokenizer.encode("Gravity is", return_tensors="pt").to(device)
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

Evaluations

Metrics	SmolLM2-MoE-214M-A135M	SmolLM2-135M
HellaSwag	44.1	43.1
ARC Challenge	30.7	29.5
PIQA	69.3	68.2
CommonsenseQA	19.7	19.5
TriviaQA	5.8	4.9
Winogrande	54.1	52.7
OpenBookQA	36.2	32.6

lm-eval-harness was used to conduct the evals. The baseline numbers may differ slightly from the SmolLM paper as they use LightEval instead.