Views
No views yet
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4repo = "sankalphs/noir-verdict-nemotron-4b-merged"
5tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
6model = AutoModelForCausalLM.from_pretrained(
7 repo, torch_dtype=torch.bfloat16, trust_remote_code=True,
8).cuda().eval()enable_thinking=False baked in. The system prompt for an active
interrogation is built by engine/prompts.py:build_system_prompt(...).1messages = [
2 {"role": "system", "content": "You are Greta Lindholm, junior continuity writer at WJBK. ..."},
3 {"role": "user", "content": "Where were you at the time of the theft?"},
4]
5text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)n_ctx ≥ 4096temperature 0.6–0.7, top_p 0.9–0.95max_new_tokens 180–280 per turn<|im_end|>nvidia/cuda:12.8.1-devel-ubuntu22.04 + Python 3.13torch>=2.8.0, triton>=3.4.0, unsloth_zoo[base] @ git+https://github.com/unslothai/unsloth-zoo, unsloth[base] @ git+https://github.com/unslothai/unsloth, --torch-backend=cu128causal-conv1d==1.6.2.post1 and mamba-ssm==2.3.2.post1 compiled from source with --no-build-isolation, CC=gcc, CXX=g++ (no prebuilt cu128 + Py3.13 wheel exists)SFTTrainer, packing, bf16, Unsloth LoRA (r=16, alpha=32, lr=2e-4 cosine, bs=2 grad_accum=8, 240 steps)train/modal_finetune.py--n-gpu-layers 99)| case | suspect | personality | truth_mode | failure_flags |
|---|---|---|---|---|
| 0 | Greta Lindholm | nervous | lie | none |
| 37 | (37, 1) | helpful | partial_truth | none |
| 113 | (113, 2) | arrogant | truth | none |
| 241 | (241, 3) | evasive | deflect | none |
| 497 | Greta Lindholm | nervous | lie | none |
<think> blocks, no overlong generations.