Tinychatbot-1b-v1.0 (Aditional Info)
tinychatbot-1b-v1.0 is a QLoRA finetune of Llama3.2-1b-Instruct, it was finetuned on the dolly-databricks. It is a proof of concept, as the final tinychatbot will be trained on dolly databricks and a custom personality dataset!
It was trained on Google Colab, using unsloth Studio, with QLoRA. It is lightweight (808MB), and has a 128k token context-length.
It has 1 Billion parameters. Other models will be coming soon, such as:
- Tinychatbot-v1.1
- Tinychatbot-v.1.2
Other types of models may include:
- Vision (if possible)
- Image models
