You can deploy this model and start earning money today!

yujiepan/
microllama-0.06B

Name: microllama-0.06B
Availability: InStock

OuteAI/Lite-Oute-1-65M-Instruct

finetune

apache-2.0

Views

No views yet

yujiepan/microllama-0.06B

This is the same model as OuteAI/Lite-Oute-1-65M-Instruct but is converted in FP16.

It is a small pretrained model that can do text generation. Very useful for algorithm development / debugging.

Special thanks to the original author OuteAI for the hard work and contribution.

This repo is just a backup for myself. If you find this model useful, consider using the original repo instead.

Evaluation

bash

lm_eval --model hf \
  --model_args pretrained=yujiepan/microllama-0.06B,max_length=2048,dtype="<dtype>" \
  --tasks wikitext \
  --device cuda:0 \
  --batch_size 1

Model dtype	Word perplexity
FP32	59.1905
BF16	59.1187
FP16	59.1902

Tested on A100 with lm-eval==0.4.7.