Model Card for Model ID
Model Details
GPT-Neo 1.3B model, fine-tuned on math-based prompts using ORPO.
Model Description
Evaluated on mmlu elementary mathematics dataset using Eleuther evaluation harness
| Tasks | Version | Filter | n-shot | Metric | | Value | | Stderr |
|---|
| elementary_mathematics | 1 | none | 0 | acc | ↑ | 0.2778 | ± | 0.023 |
This is a slight improvment over the base model, which has an accuracy of 0.2751 ± 0.023. However, this improvement in accuracy is within variance.
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
Uses
As a starting point for questions regarding math concepts. Actual calculations and values may not be reliable.
Out-of-Scope Use
Not to be used as a replacement to learn actual math.
Bias, Risks, and Limitations
This model is unlikely to produce accurate numerical results.
Training Details
Training Data
Training Procedure
1000 entries were randomly sampled. This subsample was further split 90/10 into train/test.
Preprocessing [optional]
Entries were split into prompt, chosen, rejected triplets. Maximum length for each of these entries was limited to 2048 characters.
Training Hyperparameters
- Training regime: [More Information Needed]
