Model:
This is a LLM trained on a dataset of DIBT_10k_prompts.
Evaluation:
| Tasks | Version | Filter | n-shot | Metric | | Value | | Stderr |
|---|
| hellaswag | 1 | none | 0 | acc | ↑ | 0.2872 | ± | 0.0045 |
| | none | 0 | acc_norm | ↑ | 0.3082 | ± | 0.0046 |
Passed argument batch_size = auto:4.0. Detecting largest batch size
Determined largest batch size: 64
Passed argument batch_size = auto:4.0. Detecting largest batch size
Determined largest batch size: 64
hf (pretrained=EleutherAI/pythia-160m,revision=step100000,dtype=float), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto:4 (64,64,64,64,64)
