You can deploy this model and start earning money today!

debajyotidasgupta/
eu-halt-deepseek-v2-lite-chat

Name: eu-halt-deepseek-v2-lite-chat
Availability: InStock

hallucination-detection

epistemic

feature-extraction

deepseek-ai/DeepSeek-V2-Lite-Chat

finetune

apache-2.0

Views

No views yet

EU-Halt heads for `deepseek-ai/DeepSeek-V2-Lite-Chat` (`frontier-panel, BOUNDARY regime`)

Lightweight epistemic-uncertainty detector: K=4 prediction heads sharing the frozen deepseek-ai/DeepSeek-V2-Lite-Chat trunk. Configuration: mid_dim=128, K=4 (frontier-panel, BOUNDARY regime).

Calibrate the sign before deploying. The direction of the disagreement signal is trunk-family-specific: on some families it rises on out-of-distribution input, on others (Llama-70B-class, gpt-oss-120B) it falls. Score ~50 known-ID and ~50 known-OOD prompts once and check which direction separates. Details: the paper and repo below.

Paper: When Uncertainty Lies (CAISc 2026, oral) · Code: github.com/debajyotidasgupta/eu-halt

Quick start

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from eu_halt import attach

model = AutoModelForCausalLM.from_pretrained(
    "deepseek-ai/DeepSeek-V2-Lite-Chat", torch_dtype=torch.bfloat16,
).to("cuda").eval()
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V2-Lite-Chat")

uncertainty = attach(
    model,
    heads_repo="debajyotidasgupta/eu-halt-deepseek-v2-lite-chat",
    mid_dim=128,
)
print(uncertainty("Who founded Quora in 2008?", tokenizer))
# Higher = more uncertain.

Files in this repo

model.safetensors — the K=4 head weights (preferred format; config embedded as metadata).
config.json — head geometry: num_heads, mid_dim, source_layers, base_model, dims.
heads_final.pt — the original torch checkpoint (kept for backward compatibility).
heads_step{500,1000,1500,2000,2500}.pt — intermediate checkpoints (where uploaded).
source_layers.json — the K=4 trunk-layer indices the heads read from.
history.json — per-step loss + disagreement + GPU stats.

Training

Dataset: HuggingFaceFW/fineweb-edu (streaming).
~2-5M tokens, batch_size 2-4, seq_len 512, ~2000-2500 steps.
AdamW (lr 3e-4 to 5e-4), 100-200 warmup steps.
K=4 heads, mid_dim=128, training_noise_std=0.01, dropout=0.1 (or both 0 for quiet variants).
Single GPU (~10-15 min on RTX A5000/A6000/L40S).

Evaluation

OOD AUROC (id vs ood), 420 samples total:

Signal	AUROC	95% CI
disagreement	0.5451	[0.4612, 0.6326]
entropy	0.6012	[0.5394, 0.6607]
last_token_unc	0.1663	[0.1076, 0.2308]
mahalanobis	0.3160	[0.1703, 0.4876]
targ_margin	0.1685	[0.1103, 0.2333]
etc_trend	0.5078	[0.4426, 0.5724]
llm_check	0.1740	[0.1104, 0.2564]
mc_dropout	0.4392	[0.3606, 0.5138]
rauq	nan	[nan, nan]
p_true	nan	[nan, nan]
semantic_entropy	nan	[nan, nan]
semantic_entropy_nli	nan	[nan, nan]
eigenscore	nan	[nan, nan]

Best signal: entropy

Intended use

Hallucination flagging at inference time (score before / during generation).
Dynamic-RAG gating (retrieve iff uncertainty > τ).
Selective prediction / risk-coverage trade-offs.
Token-level uncertainty visualization via uncertainty.per_token(text, tokenizer).

Limitations

No fine-tuning of the trunk — only the auxiliary heads are trained.
Heads are trained on web text. Specialized domains (medical, legal) may need a domain-specific recalibration.
For Gemma's 256k vocab, head output projection is ~70-100M params per head — still small relative to the trunk.

License

Apache-2.0 for the heads. The trunk model deepseek-ai/DeepSeek-V2-Lite-Chat retains its own license (Qwen3 / Llama-3 / Phi / Gemma).

Citation

bibtex

@inproceedings{dasgupta2026euhalt,
  author    = {Dasgupta, Debajyoti and Mondal, Arijit and Chakrabarti, Partha P.},
  title     = {When Uncertainty Lies: How Model Scale and Layer Geometry Quietly
               Invert the Meaning of Internal Disagreement in Large Language Models},
  booktitle = {1st Conference For AI Scientists (CAISc)},
  year      = {2026},
  url       = {https://huggingface.co/debajyotidasgupta/eu-halt-deepseek-v2-lite-chat},
}

debajyotidasgupta/eu-halt-deepseek-v2-lite-chat

EU-Halt heads for deepseek-ai/DeepSeek-V2-Lite-Chat (frontier-panel, BOUNDARY regime)