🔄 Early Stopping: Prevents overfitting with validation-based early stopping
📊 Progress Tracking: Real-time training progress with tqdm
🚀 Quick Start
Prerequisites
pip install torch tqdm
Training the Model
Prepare your training data: Place your text files in the training_corpora/ folder
Start training:
python AgGPT21.py
The model will automatically:
Load all .txt files from training_corpora/
Build vocabulary from your data
Train with validation split and early stopping
Save the trained model as AgGPT21.pt
Interactive Chat
Once trained, start chatting with your model:
python chat.py
📁 Project Structure
AgGPT-21-2/
├── banner.png # Project banner image
├── AgGPT21.py # Main training script
├── chat.py # Interactive chat interface
├── README.md # This file
├── AgGPT21.pt # Trained model (generated after training)
└── training_corpora/ # Training data folder
├── corpora_000.txt # Training file 1
├── corpora_001.txt # Training file 2
├── ... # More training files
└── corpora_041.txt # Training file N
⚙️ Configuration
Model Hyperparameters
Parameter
Default
Description
SEQ_LEN
64
Sequence length for training
EMBED_SIZE
128
Embedding dimension
HIDDEN_SIZE
128
GRU hidden dimension
NUM_LAYERS
1
Number of GRU layers
DROPOUT
0.2
Dropout rate
Training Parameters
Parameter
Default
Description
BATCH_SIZE
8
Training batch size
EPOCHS
6
Maximum training epochs
LR
2e-3
Learning rate
WEIGHT_DECAY
1e-4
L2 regularization
CLIP_NORM
1.0
Gradient clipping
Generation Settings
Parameter
Default
Description
TEMPERATURE
0.9
Sampling temperature (0.1-2.0)
TOP_K
50
Top-k sampling limit
TOP_P
0.9
Nucleus sampling threshold
GENERATE_LENGTH
200
Default generation length
🎮 Chat Commands
In the interactive chat mode, you can use these commands:
Basic Chat: Just type your message
quit/exit/bye: End the conversation
help: Show available commands
clear: Clear the screen
model: Display model information
temp X: Set temperature (e.g., temp 0.8)
length X: Set response length (e.g., length 150)
🧪 Example Usage
Training Example
python
1# Train the model (automatic multi-file loading)2python AgGPT21.py
Output:
Found 42 training files
Reading corpora_000.txt...
Reading corpora_001.txt...
...
Total words loaded: 2,847,392
Vocabulary size: 30,000
Tokens used: 1,000,000 | device=mps
Model params: 4,099,200
Train batches per epoch: 1,562 | Val batches: 79
Epochs: 100%|████████████| 6/6 [05:23<00:00, 53.92s/it, train=2.1847, val=2.3456]
Saved AgGPT21.pt
Chat Example
👤 You: Tell me about artificial intelligence
🤖 AgGPT-21: Artificial intelligence is a fascinating field that focuses on creating systems capable of performing tasks that typically require human intelligence. These systems can learn from data, recognize patterns, make decisions, and solve complex problems. AI has applications in many areas including natural language processing, computer vision, robotics, and machine learning...
🔧 Advanced Usage
Custom Vocabulary Size
MAX_VOCAB = 50000 # Increase vocabulary size
Training on Subset of Data
python
1DATA_PERCENT =0.5# Use only 50% of available data2MAX_TOKENS =500_000# Limit to 500k tokens
Multi-GPU Training
python
1# The model automatically detects and uses available accelerators:2# - CUDA (NVIDIA GPUs)3# - MPS (Apple Silicon)4# - CPU (fallback)
