You can deploy this model and start earning money today!

oldman-dev/
tis-v8b-hard-anchor

Name: tis-v8b-hard-anchor
Availability: InStock

mistralai/Mistral-7B-v0.3

finetune

mit

endpoints_compatible

Views

No views yet

Token Importance Scoring (TIS) v8b - Hard Anchor Final

This checkpoint contains the Token Importance Scoring (TIS) components trained with hard-anchor forcing and tuned stability weights for publication results.

Model Description

This is the publication checkpoint for TIS v8b, featuring optimized hard-anchor forcing and multi-objective loss tuning. It demonstrates strong performance on low-budget scenarios (25% cache budget) with 82% NIAH accuracy.

Key Features:

✅ 82% NIAH accuracy at 25% cache budget (100% evidence survival)
✅ Hard-anchor forcing value: 1.0 (deterministic query/evidence preservation)
✅ Multi-objective loss: ranking + retrieval + stability
✅ Includes benchmark results (NIAH Hard)

Performance

NIAH Hard Benchmark Results

Cache Budget	Accuracy	Evidence Survival
10%	8.0%	19.3%
25%	82.0%	100.0%
50%	68.0%	-
75%	40.0%	69.8%

Comparison vs Heuristic (H2O-style):

At 25% budget: 82% vs 14% (5.9x improvement)
At 50% budget: 68% vs 28% (2.4x improvement)

Training Configuration

Base Model: mistralai/Mistral-7B-v0.3
Approach: V8 Hard Anchor Restored
Training Steps: 2,000

Loss Weights:

Ranking Loss: 1.0
Retrieval Loss: 2.0
Stability Loss: 0.5

Hyperparameters:

Learning rate: 1e-3
Gradient accumulation: 4
Hard negative margin: 0.5
Training budgets: [0.25, 0.5, 0.75]
Hard-anchor force value: 1.0

Model Architecture

This checkpoint contains:

ImportanceUpdateHead: Hard-anchor forcing importance predictor
Importance Embedding: Token-level importance embeddings

Components:

python

{
  'importance_embedding': dict,  # Token importance embeddings
  'importance_head': dict,       # Hard-anchor predictor (7 keys)
}

Note: This checkpoint has 7 keys in importance_head (vs 6 in stage3), indicating additional hard-anchor forcing components.

Usage

Installation

bash

git clone https://github.com/nitroxido/token-importance-scoring
cd token-importance-scoring
python -m venv .venv
source .venv/bin/activate
pip install -e .

Load Checkpoint

python

from token_importance.model.importance_head import ImportanceUpdateHead
import torch

# Load TIS components
checkpoint = torch.load('tis_components.pt', map_location='cuda')

# Extract components
importance_head_state = checkpoint['importance_head']
importance_embedding_state = checkpoint['importance_embedding']

print(f"Importance head keys: {importance_head_state.keys()}")

Evaluate on NIAH Hard Benchmark

bash

# Download checkpoint first
hf download oldman-dev/tis-v8b-hard-anchor --local-dir checkpoints/v8b_hard_anchor

# Run evaluation
python scripts/eval_niah_hard.py \
  --learned-checkpoint checkpoints/v8b_hard_anchor \
  --budgets 0.25 0.5 0.75 \
  --num-tests 50 \
  --context-tokens 2048 \
  --device cuda \
  --seed 42

Training Details

Training Objective:

Loss = ranking_loss + 2.0 * retrieval_loss + 0.5 * stability_loss

Hard-Anchor Forcing:

Forces importance scores of 1.0 for query tokens and evidence tokens
Ensures deterministic preservation of critical information
Prevents catastrophic forgetting in low-budget scenarios

Training Data: Synthetic retrieval tasks with hard negatives

Intended Use

Primary Use Cases:

Low-budget KV cache compression (25-50% budgets)
Retrieval-focused applications requiring evidence preservation
Scenarios where query context must be preserved

Best Performance:

25% cache budget: 82% NIAH accuracy with perfect evidence survival
Ideal for extreme memory-constrained deployments

Limitations:

Performance degrades at very low budgets (10%)
Lower performance at high budgets compared to stage3-ert
Optimized for retrieval tasks, not general generation

Comparison with Other Checkpoints

Checkpoint	NIAH @ 25%	NIAH @ 50%	Best Use Case
tis-v8b-hard-anchor (this)	82%	68%	Low-budget retrieval
tis-stage3-ert	98%	100%	General retrieval + LITM
tis-stage1-oracle	100%	100%	Oracle baseline

Citation

If you use this checkpoint, please cite:

bibtex

@software{token_importance_scoring_2026,
  title={Token Importance Scoring: Learned KV Cache Compression for Long-Context LLMs},
  author={Token Importance Scoring Contributors},
  year={2026},
  url={https://github.com/nitroxido/token-importance-scoring}
}

License

MIT License - See LICENSE

Acknowledgments

Training compute sponsored by GPU-Action.

More Information

Repository: https://github.com/nitroxido/token-importance-scoring
Documentation: See REPOSITORY-OVERVIEW.md and REPRODUCIBILITY-GUIDE.md in the repository
Related Checkpoints:
- tis-stage3-ert - Main ERT checkpoint (100% NIAH @ 50%)
- tis-stage1-oracle - Oracle-labeled baseline