Model	Parameters	Training iters	Final Train Loss	Quality	Example Chat
Pin-5M	5.37M	1000	3.170788	Very Poor	Yes, a bright day is shining and makes everything have a good day a lot.
Pin-10M	10.06M	1500	2.562048	Very Poor	That sounds nice, I agree, it is nice to talk about new ideas.
Pin-15M	14.84M	1500	2.358367	Low	It is hard to see your plans when you want to enjoy the day.
Pin-20M	21.03M	1500	2.217588	Medium	Yes, sunny days are wonderful! I love hearing about the sunshine and the sun's shining on.
Pin-25M	26.76M	1500	2.139837	Medium	Sunny days make everything look brighter, especially with a nice friend who cares.
Pin-Ultra-25M	26.76M	8000	1.588281	Hight	Sunny days are wonderful, they really lift our spirits and make us smile.

* All models were prompted with What is the weather like today?.

🚀 We recommend using Pin-Ultra-25M.

Training

Training data

We trained on starhopp3r/TinyChat and used the gpt-2 tokenizer.

Training code/scripts

You can find the full training code for the Pin Model Series in this repo.
Tip: If you want to train one of these models yourself, make sure to adjust the model config like this:

Model	n_layer	n_head	n_embd	n_inner
Pin-5M	4	8	96	384
Pin-10M	6	8	160	640
Pin-15M	8	8	208	832
Pin-20M	10	8	256	1024
Pin-25M	12	12	288	1152
Pin-Ultra-25M	12	12	288	1152

Have fun :D

Training details

We trained all these models in ~30 minutes on Kaggle 2xT4 GPU in a Kaggle Session (except for Pin-Ultra-25 - it trained for ~4 hours on Kaggle 2xT4).
So you are able to easily recreate all of the Pin model without having to launch a 8xH100 cluster 😂

How to use the model

You can easily use the favorite model of the Pin series like this:

Download use.py from this repo.
Adjust the subfolder argument here:
answer = run_pin_inference(user_query, model_id="LH-Tech-AI/Pin", subfolder="Pin-Ultra-25M") # use your favorite model here, e.g. "Pin-25M" or "Pin-15M"...
Adjust the input prompt here:
user_query = "What is the weather like today?" # insert your prompt here
Launch it with Torch installed in Python3.
Have fun :D

Acknowledgements...

...to:

HF Transformers
Kaggle for the 2xT4 GPU
starhopp3r for his TinyChat dataset

LH-Tech-AI/Pin

Welcome to 🤏 Pin Model Series

Models overview