PREDY 1.1
Predy 1.1 is a finetuned version of gpt-2 available, at the moment, in the English language.
His goal is to predict the place of domotic commands.
Usage
This model is mainly made for experimental purposes, but you can still use it with pytorch model loading.
To try using this model you can simply run the kaggle notebook "Inference for predy V1", in this notebook there are all the requirements to inference our Predy model.
System
Predy series 1 is still an experiment but in the future it will be a fundamental part of light DLMs.
Implementation requirements
If you want to train this model with a custom dataset we recommend using this hyperparameters:
- Optimizer: AdamW
- Learning rate: 0.0005
- dataset lenght in tokens: >10000
- epochs: 20
With this hyperparameters, you can reach up to 99% accuracy.
The inference requirements are really low since it only has 124M parameters, anyway here are the minimum requirements to inference Predy 1:
- ram > 2 GiB dedicated for the model (int4)
- CPU, even a raspberry Pi 5 can run it smoothly
Model Characteristics
Data Overview
The Dataset was sinthetically generated with AI, to ensure quality the model used was Gemini 3.1 Pro .
A line of the dataset looks like this:
"turn on the basement lights",basement
As Predy is still an experiment it uses a csv structured dataset, in future versions the structure used will be jsonl based.
README in progress...
