Views
No views yet
batch_size: 512 context_length: 1024 learning_rate: 2e-4 schedule: cosine with 10% warmup from 0 to 2e-4, cooldown to 0
pytorch transformers einops
