• Model creator: https://huggingface.co/pszemraj/
• Original model: https://huggingface.co/pszemraj/pythia-31m-simplepile-lite-2048-scratch-2e/

Original model description:

• generated_from_trainer metrics:
• accuracy inference: parameters: max_new_tokens: 64 do_sample: true repetition_penalty: 1.1 no_repeat_ngram_size: 5 guidance_scale: 1.01 eta_cutoff: 0.001 widget:
• text: My name is El Microondas the Wise and example_title: El Microondas
• text: A meme is example_title: meme
• text: >- Barack Obama nominated Hilary Clinton as his secretary of state on Monday. He chose her because she had example_title: Coreference resolution
• text: >- On a shelf, there are five books: a gray book, a red book, a purple book, a blue book, and a black book example_title: Logic puzzles
• text: >- The two men running to become New York City's next mayor will face off in their first debate Wednesday night example_title: Reading comprehension pipeline_tag: text-generation license: apache-2.0 datasets:
• pszemraj/simplepile-lite

---

BL-pythia-31m-simplepile-lite-2048-scratch

• Loss: 3.9891
• Accuracy: 0.3498

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

• learning_rate: 0.0005
• train_batch_size: 2
• eval_batch_size: 1
• seed: 80085
• gradient_accumulation_steps: 64
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.99) and epsilon=1e-07
• lr_scheduler_type: inverse_sqrt
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 2.0

Training results

Framework versions

• Transformers 4.33.1
• Pytorch 2.2.0.dev20230907+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3

Open LLM Leaderboard Evaluation Results