You can deploy this model and start earning money today!

Adolfo-GM/
AgmGPT-3.5

Name: AgmGPT-3.5
Availability: InStock

Adolfo-GM/AgmGPT-3

finetune

mit

Views

No views yet

AgmGPT-3.5

AgmGPT-3.5 is a compact 25.7M params conversational GPT model trained on a synthetic dialogue dataset. It is designed for short, helpful, conversational responses with a consistent AGM assistant identity.

Model Details

The checked-in training config targets a compact GPT model. The active architecture is stored in the checkpoint at out/agm_3.5.pt under model_args, and the tokenizer metadata is stored in data/textfile_word/meta.pkl.

The included textfile_word tokenizer metadata currently reports:

Vocabulary Size: 455
Special Tokens: <|user|>, <|ai|>, <|eos|>
Tokenizer Metadata: data/textfile_word/meta.pkl
Default Checkpoint: out/agm_3.5.pt

Tokenization

AgmGPT-3.5 uses a word-level tokenizer with character fallback:

special tags like <|user|>, <|ai|>, and <|eos|> are single tokens
words are matched as word tokens when they exist in meta.pkl
digits, whitespace, punctuation, and symbols are tokenized separately
unknown words fall back to individual character tokens

The vocabulary includes all individual characters that existed in the original prepared training text. That means new words usually map correctly by falling back to characters. A truly new character that is not in meta.pkl cannot be added during fine-tuning without changing the model shape, so it should be replaced in the fine-tune data or handled by retraining from scratch with a rebuilt vocab.

Usage

Chat

To chat interactively:

python chat.py

Training

To prepare the word-level data:

python data/textfile_word/prepare.py

To train using the word-level config:

python train.py data/train_textfile_word.py

Or with manual overrides:

python train.py data/train_textfile_word.py --batch_size=32 --compile=False

Fine-Tuning

To fine-tune the existing checkpoint on newer data:

python fine_tune/fine_tune.py

The fine-tune script reads fine_tune/data.txt, tokenizes it with the existing data/<dataset>/meta.pkl vocab from the checkpoint config, and overwrites the same checkpoint path when training finishes.

Useful overrides:

bash

python fine_tune/fine_tune.py --max_iters=100 --learning_rate=5e-5 --batch_size=4
python fine_tune/fine_tune.py --data_file="fine_tune/data.txt" --ckpt_path="out/agm_3.5.pt"

Sampling

To generate raw text samples:

python chat.py sample

You can also override sampling parameters:

python chat.py sample --num_samples=5 --max_new_tokens=200

Training Run