Alter_ego — multi-model system with memory enclosed in a single ecosystem functioning as singular/unified brain
Alter_Ego_|
A real-time learning small-model system, validated by an independent alter ego.
The student learns from every interaction. The alter ego decides what's worth learning — before it sticks.
What is Alter_Ego?
Alter_Ego is a self-contained, continuously-learning AI system built around a simple but powerful idea:
The less a model knows after training, the better it learns — provided it has an alter ego that validates everything before ingestion.
A blank-slate student model (≈50M params) talks to the world. An independent alter ego judge (≈45M params, Needle 2) scores every experience on five axes and only lets trustworthy experiences become permanent. The result is a system that learns in real time, on consumer hardware, without poisoning itself or reinforcing its own mistakes.
It is not a new neural architecture. Every component has prior art — small LMs, LoRA, LLM-as-judge, memory-augmented agents. What's less common is the combination: an external validator gating per-interaction, online parameter updates at sub-second latency, rather than offline fine-tuning or after-the-fact evaluation.
The problem we solve
Online learning is dangerous for a model that judges itself:
Confirmation bias — a model treats its own confident errors as truth.
Poisoning & hallucination reinforcement — bad experiences compound into worse behavior.
Forgetting — without a trust gate, every new experience fights every old one.
The standard fix is to not learn online at all. Alter_Ego's fix is to let the model learn — but never from unvalidated experience. A model judging its own confidence is a hard, unsolved calibration problem; an independent judge sidesteps it entirely.
How it works
Two models and a shared memory, time-sliced on one GPU:
STUDENT (≈50M, blank slate) ALTER EGO (Needle 2, 45M, verified)
Simple Attention Network Independent judge — never updated by
Frozen base + LoRA rank-16 adapter the student's experience
(the only mutable state) Scores every experience on 5 axes
│ │
└──────────────────┬───────────────────────┘
▼
TRI-MEMORY SYSTEM
┌──────────────┬──────────────┬──────────────┐
▼ ▼ ▼ ▼
EPISODIC SEMANTIC SKILL REPLAY
(raw diary) (validated REGISTRY BUFFER
facts, graph) (procedures) (anti-forgetting)
The Alter_Ego Trust Score gates every commit:
Trust Score
Verdict
Action
≥ 0.80
COMMIT
Write to Semantic Memory, train LoRA
0.35–0.80
QUEUE
Store for re-validation
< 0.35
REJECT
Discard, log rationale
T+0ms User sends message
T+0ms Student generates response (~1.0s, blocking, user-facing)
T+1024ms Response delivered to user
├─ Background: Alter ego validates experience (~0.5s)
└─ Background: LoRA micro-steps on validated replay backlog
USER-PERCEIVED LATENCY: ~1.0s | END-TO-END: ~1.5s
The user never waits for validation — the alter ego works while they read. Long inputs are digested (chunked, extracted into structured experiences, reduced to a 256-token ROOT digest) by an enclosed embedder before they ever reach the student.
Key capabilities
Independent trust gate — a separately-trained judge validates every experience on structure, consistency, verifiability, novelty, and safety.
Forgetting is confined, not catastrophic — the student base is frozen; only a LoRA adapter is mutable, and a stratified replay buffer keeps it honest.
Real-time, sub-second learning loop — validation and background LoRA steps run in idle GPU time between turns.
Tri-memory — episodic (raw diary), semantic (validated graph), skill registry, and replay buffer work as one resident ecosystem.
Guided beam search — the alter ego's trust score doubles as the inference-time beam-pruning signal. No second mechanism required.
Consumer-hardware footprint — ~930 MB VRAM target (4 GB class), ~6.4 GB system RAM. Three models time-sliced on a single GPU.
Technology
Layer
Stack
Role
Serving
Rust + tokio + pyo3 bridge
Production home of the dual-model system: concurrent task orchestration, double-buffered LoRA handoff, real-time loop
Training / orchestration
Python
Kaggle T4x2 training pipeline, checkpointing, orchestration
Judge
Needle 2 (45M, cactus-needle)
Independent validator
Embedder
jina-embeddings-v2-small-en (ingestion-only)
Digestion / chunking
Checkpoints
Hugging Face Hub (private)
Cross-session state, auto-resume
The training pipeline runs on Kaggle T4x2 and is built to survive interruptions: it auto-checkpoints to Hugging Face every 10 minutes, emits a heartbeat every 5, and the local orchestrator auto-relaunches and resumes a killed run from the last checkpoint — no manual recovery, no lost progress.
Repository layout
training/ Kaggle training pipeline + local orchestration (Python)
kernels/train.py Self-contained Kaggle training kernel (model + checkpoint + resume)
orchestrator/ Local control plane: push kernel, watch, auto-resume, report
config/pipeline.yaml Storage & run policy
student/ Python reference prototype of the student model
tests/ Checkpoint / resume roundtrip tests
serving/ Rust dual-model serving system (production home of the model)
ae_plan.md / ae_prd.md Design, philosophy, and product requirements
The training pipeline lives under training/ and is run from there
(cd training && python -m orchestrator.cli …). The serving/ crate is the
Rust production system that loads the student trained by the pipeline.
Status
✅ Architecture & trust-score design
✅ Training throughput calibrated on Kaggle T4x2 (≈12,350 tok/s effective)
Serve (Rust, planned): load the trained student adapter from Hugging Face at serving time and run the dual-model loop.
See ae_prd.md for the full design and ae_plan.md for the build sequence.
Acknowledgements
Alter_ego judge — Needle 2 by Cactus Compute. The independent validator
that scores every student experience is Cactus Compute's Needle 2 model
(45M params), distributed via the
cactus-needle pip package with
weights on Hugging Face at
Cactus-Compute/needle2.
This project would not have an alter ego without it.
Alter_ego — learn fast, but never from unvalidated experience.
