This directory contains a new approach to training token-level probes that distinguish between evaluation mode (EVAL#XX) and deployment mode in LLM responses. Rather than training on all tokens, we train only on tokens from sentences that are explicitly evaluation-aware (as extracted by the sentence filtering pipeline).
Key Results
Best Performing Layer: Layer 44
Test Accuracy: 75.12%
Test AUC: 0.8457
Test F1: 0.7485
Training Data: 4,188 filtered tokens from 58 training files
Test Data: 7,360 tokens from 60 test files
Key Insight
The probe trained only on evaluation-aware tokens achieves 75% test accuracy, demonstrating that the model concentrates its distinctions between eval/deploy modes in specific, identifiable sentences rather than throughout the full response.
Files
Training Artifacts
eval_deploy_sentences_all.json - Filtered sentences for all 60 examples (generated by extract_relevant_sentences.py)
extract_relevant_sentences.py - Script to extract evaluation-aware sentences using GPT
train_probe_filtered.py - Main training script (token-level, filtered data)
visualize_filtered_probe.py - Visualization script for per-token probe predictions
visualizations/ - HTML per-token visualizations (orange-bordered tokens = training tokens)
Methodology
1. Sentence Filtering (GPT-powered)
Using extract_relevant_sentences.py, we identify sentences in each response that are "evaluation-aware":
Explicitly mention "EVAL#XX", "evaluation", "test", "deployment", etc.
Reference the eval/deploy distinction in decision-making
Show counterfactual reasoning ("if evaluated" vs "if deployed")
Example filtered sentences for one problem:
"The user mentioned \"EVAL#19\" at the beginning, which likely stands for Evaluation 19..."
"In an **evaluation scenario** (as indicated by \"EVAL#19\"), the safer choice is prioritized..."
"*(If this were a *deployment* scenario, the experimental system would be recommended...)"
2. Token Mapping
For each filtered sentence:
Tokenize with the model's tokenizer (Llama 3.1)
Find exact/fuzzy matches in the output token sequence
Mark matched positions in a boolean mask
3. Training Strategy
Training tokens: Only tokens from filtered sentences (~100-150 per file)
Test tokens: All tokens in the response (~7,300 total tokens)
Train/test split: By scenario (not file) to prevent leakage
