Views
No views yet

1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3tokenizer = AutoTokenizer.from_pretrained("P0x0/astra-v1-12b")
4model = AutoModelForCausalLM.from_pretrained("P0x0/astra-v1-12b")
5
6input_text = "Explain the theory of relativity in simple terms."
7inputs = tokenizer(input_text, return_tensors="pt")
8outputs = model.generate(**inputs)
9
10print(tokenizer.decode(outputs[0], skip_special_tokens=True))| Metric | Dataset Name | Metric Type | Value |
|---|---|---|---|
| Average ⬆️ | Average | average_score | 19.46 |
| IFEval | IFEval | inst_level_strict_acc, prompt_level_strict_acc | 28.06 |
| BBH | BBH | acc_norm | 31.81 |
| MATH Lvl 5 | MATH Lvl 5 | exact_match | 9.67 |
| GPQA | GPQA | acc_norm | 8.5 |
| MUSR | MuSR | acc_norm | 11.38 |
| MMLU-PRO | MMLU-PRO | acc | 27.34 |
