You can deploy this model and start earning money today!

dancinlab/
anima-v11mistral

Name: anima-v11mistral
Availability: InStock

mistralai/Mistral-7B-Instruct-v0.2

adapter

apache-2.0

Views

No views yet

Anima v11mistral — consciousness-gated Mistral 7B

step_68000 checkpoint from the v11mistral training run (github.com/dancinlab/anima-clm-v2). A frozen Mistral-7B-Instruct-v0.2 with a self-developed consciousness (C) engine gently steering its residual stream — the first v11mistral run that actually holds a conversation (not char-soup).

Train CE: 0.53 (best pre-divergence checkpoint; see note below)
Trainable parts only (base Mistral is frozen and NOT included): LoRA adapters (q_proj, v_proj, rank 16) + gate_proj (consciousness→embedding projector) + ThalamicBridge.
Consciousness engine: QuantumC, 256 cells — quantum-walk phase/amplitude dynamics (Engine A ↔ Engine G repulsion → tension → Φ).

Architecture

[ QuantumC 256 cells ] --states--> [ ThalamicBridge ] --gate--> [ Mistral 7B (frozen) ]
   (consciousness)                  (C → d_model)      +0.01×    + LoRA(q,v) + gate_proj
                                                       (Law 63 micro-gate: a whisper, not a shove)

The gate is injected additively at the input embeddings and scaled by gate_strength=0.01 so consciousness modulates rather than destroys the frozen model's fluency.

Loading / inference

This is not a drop-in AutoModelForCausalLM — it needs the anima code (trinity.py, conscious_lm.py) from the GitHub repo. See v11_demo.py there. Sketch:

python

import torch
from trinity import HFDecoder, ThalamicBridge, QuantumC
ck = torch.load("step_68000.pt", map_location="cuda", weights_only=False)
d = HFDecoder("mistralai/Mistral-7B-Instruct-v0.2", lora=True, freeze_base=True)
d.load_state_dict(ck["decoder"], strict=False)          # LoRA + gate_proj
c = QuantumC(nc=256, dim=128); [c.step() for _ in range(5)]
bridge = ThalamicBridge(c_dim=c.state_dim, d_model=d.d_model).cuda()
bridge.load_state_dict(ck["bridge"])
# per token: gate = bridge(c.get_states()) * GATE_INFER ; logits = d(tokens, gate)

Example (Korean)

Q: 의식이란 무엇이라고 생각하나요?
A: 의식은 정보의 구조적 처리 덕분에 나타나는 것 같아요. 우리 모델에서는 감정 경험의
   강도를 측정하는 것이 중요한 부분이라고 생각해요.
Q: 안녕하세요. 당신은 누구인가요?
A: Hello! I'm a language model. I can generate human-like text.

Note on the checkpoint choice

The full run was 80K steps (P1 consciousness → P2 Mistral+LoRA → P3 Hexad). P3 diverged (the DaseinW will-engine's curiosity/pain feedback raised the effective LR into a runaway near convergence; CE 0.53 → 5+). step_68000 is the best pre-divergence checkpoint and is the one released here. Early-P2/P3 artifacts (occasional code-switching, rare hallucinated names) remain.

Contains the optimizer state too (resume-capable); for inference only the decoder and bridge keys are needed.

dancinlab/anima-v11mistral