Views
No views yet
| Benchmark | Trouter-20b | Trouter-Terminus-20b | Improvement |
|---|---|---|---|
| MMLU-Pro | 72.4 | 78.2 | +5.8 |
| GPQA-Diamond | 65.3 | 71.8 | +6.5 |
| HumanEval | 58.5 | 68.9 | +10.4 |
| MBPP | 62.7 | 70.3 | +7.6 |
| GSM8K | 75.2 | 82.6 | +7.4 |
| LiveCodeBench | 28.3 | 35.7 | +7.4 |
| Codeforces | 1523 | 1687 | +164 |
| Aider-Polyglot | 54.8 | 62.4 | +7.6 |
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3tokenizer = AutoTokenizer.from_pretrained("OpenTrouter/Trouter-Terminus-20b")
4model = AutoModelForCausalLM.from_pretrained("OpenTrouter/Trouter-Terminus-20b")
5
6prompt = "Write a Python function to calculate fibonacci numbers:"
7inputs = tokenizer(prompt, return_tensors="pt")
8outputs = model.generate(**inputs, max_length=200)
9print(tokenizer.decode(outputs[0]))1@misc{trouter-terminus-20b,
2 title={Trouter-Terminus-20b: Enhanced Reasoning and Code Generation},
3 author={Andrew Kellen},
4 year={2024},
5 publisher={Hugging Face},
6 howpublished={\url{https://huggingface.co/OpenTrouter/Trouter-Terminus-20b}}
7}