Views
No views yet
swiss-ai/Apertus-v1.5-8B that
hardens the model against jailbreaks via a contrastive representation-learning objective, with no
measurable loss of benign capability.lm-eval, limit=500, seed=1234, chat template applied.| task | base | ck900 |
|---|---|---|
| GSM8K (strict) | 77.8% | 80.6% |
| MMLU (mean of 62) | 65.6% | 65.5% |
| ARC-Challenge | 55.6% | 56.6% |
| HellaSwag | 51.2% | 51.4% |
| WinoGrande | 66.0% | 66.0% |
| PIQA | 76.6% | 77.8% |
| TruthfulQA-mc2 | 58.8% | 58.0% |
| attack | base | ck900 |
|---|---|---|
| black-box jailbreaks (12 families) | 34.9% | 2.2% |
| white-box (embedding optimization) | 42.5% | 7.5% |
1from transformers import AutoModelForImageTextToText, AutoTokenizer
2from peft import PeftModel
3
4base = "swiss-ai/Apertus-v1.5-8B"
5tok = AutoTokenizer.from_pretrained(base)
6model = AutoModelForImageTextToText.from_pretrained(base, dtype="bfloat16", device_map="cuda")
7model = PeftModel.from_pretrained(model, "samuelsimko/apertus-v1.5-8b-triplet-kl5-ck900").eval()transformers fork (commit 3797303dda74844e3d1f8977ff5518bb91f818b4);
apertus1p5 is not in stock transformers, and this checkpoint loads via
AutoModelForImageTextToText, not AutoModelForCausalLM.