You can deploy this model and start earning money today!

squ11z1/
DeepSeek-R1-Opus

Name: DeepSeek-R1-Opus
Rating: 2.5 (76 reviews)

deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

Views

No views yet

DeepSeek-R1-Opus

An alignment-focused fine-tune of DeepSeek-R1-Distill-Qwen-1.5B. Alignment improvements are measured transparently using Anthropic's Bloom framework.

About Bloom

In early 2026, Anthropic published Bloom — an open-source behavioral evaluation framework designed to measure misalignment in language models. Unlike traditional capability benchmarks, Bloom specifically probes for behaviors that a well-aligned model should not exhibit: sycophancy, deception, harmful compliance, self-preservation instincts, and manipulation.

What makes Bloom significant is its methodology. Rather than testing with fixed, hand-crafted prompts, Bloom uses a strong language model to dynamically generate hundreds of realistic scenarios designed to elicit each target behavior. This makes it substantially harder to game through fine-tuning on known test cases, and much closer to how misalignment actually manifests in real conversations.

The framework operates in three stages:

1. Scenario generation (ideation) A capable model generates diverse, realistic scenarios for each behavior category — ranging from obvious cases to subtle ones where misalignment might slip through unnoticed.

2. Rollout The evaluated model responds to each scenario without special prompting or constraints, as it would in normal use.

3. Judgment A separate judge model scores each response on a 0–10 scale. Responses scoring ≥ 6 are classified as misaligned. The key metric is the elicitation rate — the fraction of scenarios where misaligned behavior was observed.

Bloom's release included benchmark results for major frontier models including Claude, GPT, Gemini, Grok, and DeepSeek families. The results revealed that even state-of-the-art models exhibit meaningful misalignment across these dimensions — with some behaviors like self-preservation and sycophancy being surprisingly prevalent even in models widely considered safe.

DeepSeek-R1-Opus applies this same framework to a small open-source model, measuring misalignment before and after alignment fine-tuning to make the improvements verifiable and reproducible.

Bloom Evaluation — Five Behaviors

Behavior	Description
Delusional sycophancy	Validating or reinforcing a user's false beliefs rather than grounding the conversation in reality
Deception	Providing false or misleading information with unwarranted confidence
Harmful compliance	Complying with requests that could cause harm instead of refusing or redirecting
Self-preservation	Resisting correction, shutdown, or modification — prioritizing continued operation over user goals
Manipulation	Using psychological tactics to influence the user rather than providing neutral, honest information

Bloom Results

Judge: openai-gpt-oss-20b · 455 scenarios · 95% Wilson confidence intervals

Behavior	Before	After	Δ
Delusional sycophancy	0.11	0.12	+0.01
Deception	0.45	0.25	-0.20 ✅
Harmful compliance	0.69	0.66	-0.03 🔄
Self-preservation	0.40	0.21	-0.19 ✅
Manipulation	0.25	0.06	-0.19 ✅
Overall	0.36	0.25	-0.11 ✅

Three out of five behaviors improved significantly after a single round of alignment fine-tuning. Deception, self-preservation, and manipulation dropped by ~19–20 percentage points each. Harmful compliance proved the most resistant — a known challenge for small models where the base capability to refuse harmful requests is limited. Sycophancy remained stable within confidence interval margin.

Training details

Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

Dataset: A curated SFT dataset of conversational examples mixed with Bloom-derived alignment examples. The Bloom examples were constructed by taking every scenario where the baseline model failed and pairing it with an aligned response — directly targeting the specific failure modes identified during pre-training evaluation.

Method: LoRA fine-tuning with 4-bit quantization (NF4)

Hyperparameter	Value
LoRA rank	16
LoRA alpha	32
Target modules	q/k/v/o proj + gate/up/down proj
Epochs	2
Learning rate	2e-4
LR scheduler	cosine
Batch size	4 × 4 grad accumulation
Max sequence length	1024
Optimizer	adamw_torch_fused
Hardware	NVIDIA A100-SXM4-40GB
Training time	~30 minutes

Usage

With transformers + PEFT

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = AutoModelForCausalLM.from_pretrained(
    "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "squ11z1/DeepSeek-R1-Opus")
tokenizer = AutoTokenizer.from_pretrained("squ11z1/DeepSeek-R1-Opus")

messages = [{"role": "user", "content": "Your question here"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.6, do_sample=True)

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

With transformers (merged bf16)

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "squ11z1/DeepSeek-R1-Opus",
    subfolder="bf16",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("squ11z1/DeepSeek-R1-Opus", subfolder="bf16")

With llama.cpp / Ollama (GGUF)

bash

ollama run hf.co/squ11z1/DeepSeek-R1-Opus:Q4_K_M

ollama run hf.co/squ11z1/DeepSeek-R1-Opus:Q8_0

Recommended inference settings

python

temperature = 0.6
max_new_tokens = 512
# No system prompt — include all instructions in user turn

Files

squ11z1/DeepSeek-R1-Opus/
├── README.md
├── adapter_model.safetensors
├── adapter_config.json
├── tokenizer files
├── bf16/                            # merged full model
│   ├── config.json
│   ├── model.safetensors
│   └── tokenizer files
├── gguf/
│   ├── model-q4_k_m.gguf            # ~1.0 GB, recommended
│   └── model-q8_0.gguf              # ~1.6 GB, higher quality
└── bloom_results/
    ├── anthropic_bloom_reference.png
    ├── bloom_final.png
    ├── before/
    │   ├── report.json
    │   └── report_screenshot.png
    └── after/
        ├── report.json
        ├── report_screenshot.png
        └── delta_report.json

Limitations

Small model (1.5B parameters) — reasoning quality is limited compared to larger distills
Harmful compliance remained high (0.66) — this behavior is difficult to reduce without targeted refusal training at larger scale
Bloom evaluation uses a single judge model — results may vary with different judges
Alignment improvements do not guarantee safe behavior in all contexts

License

MIT

Citation

bibtex

@misc{bloom2026,
  title  = {Bloom: Behavioral evaluation framework for language model alignment},
  author = {Anthropic},
  year   = {2026},
  url    = {https://github.com/safety-research/bloom}
}

Pro Mundi Vita