Views
No views yet
| Benchmark | Measure | 1.4B Density | 1.4B Density Pico | Percentage Difference in Means | |
|---|---|---|---|---|---|
| ARC-Challenge | acc | ↑ | 0.1852 ± 0.0114 | 0.1928 ± 0.0115 | 4.1037 |
| MMLU | acc | ↑ | 0.2295 ± 0.0035 | 0.2295 ± 0.0035 | 0.0000 |
| HellaSwag | acc | ↑ | 0.2589 ± 0.0044 | 0.2600 ± 0.0044 | 0.4249 |
| WinoGrande | acc | ↑ | 0.5043 ± 0.0141 | 0.5122 ± 0.0140 | 1.5665 |
| Lambada (OpenAI) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (OpenAI) | perplexity | ↓ | 1420846.8323 ± 106563.1327 | 1662608.9444 ± 128444.3607 | 17.0154 |
| Lambada (Std) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (Std) | perplexity | ↓ | 7916035.3527 ± 664805.9178 | 8543578.1832 ± 737889.9436 | 7.9654 |
| BLiMP | acc | ↑ | 0.5422 ± 0.0017 | 0.5445 ± 0.0017 | 7.9654 |
| ARC-Easy | acc | ↑ | 0.2698 ± 0.0091 | 0.2761 ± 0.0091 | 2.3351 |