You can deploy this model and start earning money today!

lebe1/
lettuceprevent-generate

transformers

custom_generate

endpoints_compatible

Views

No views yet

LettucePrevent — custom_generate

Real-time hallucination prevention during RAG generation. A token-level detector plugs into decoding through a custom LogitsProcessor. At every step the top-k candidates are scored and hallucinated tokens are penalized before they are emitted.

🔗 GitHub · Detector model

Detectors

Detector	Type	Notes
`number`	rule-based	RegEx-verifiable; rejects numbers not in the input. No checkpoint to download.
`lettuceprevent`	neural (Ettin 68M)	Flags unsupported tokens for factual claims. CUDA recommended.
`lettucedetect`	encoder	Post-hoc hallucination scoring (e.g. `KRLabsOrg/lettucedect-base-modernbert-en-v1`).

Install

bash

pip install -r custom_generate/requirements.txt
python -m spacy download en_core_web_sm

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mistral-7B-Instruct-v0.2"
tok   = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto",
                                             trust_remote_code=True)

# For instruction-tuned models, always apply the chat template
context  = "Revenue was 2400 million in 2021 and 3100 million in 2022."
question = "What is the percentage increase in revenue from 2021 to 2022?"
text     = tok.apply_chat_template(
               [{"role": "user", "content": f"{context}\n{question}"}],
               tokenize=False, add_generation_prompt=True)
inputs   = tok(text, return_tensors="pt").to(model.device)

out = model.generate(
    **inputs,
    custom_generate="lebe1/lettuceprevent-generate",   # or a local path
    trust_remote_code=True,
    tokenizer=tok,
    input_text=context,          # grounding context for the detector
    detector_type="number",      # or "lettuceprevent"
    max_new_tokens=300,
)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Key arguments

Argument	Default	Description
`tokenizer`	required	Generator tokenizer (used to decode candidates).
`input_text`	required	Grounding context validated against by the detector.
`detector_type`	`"lettuceprevent"`	`"number"`, `"lettuceprevent"`, or a `baseline-*` switch.
`skip_threshold`	`1.0`	Skip the HDM check when top-token probability exceeds this. `1.0` = always check. Important to play around with this parameter to understand its importance.
`penalty_value`	`0.0`	Score assigned to hallucinated tokens (`float('-inf')` to hard-block).
`confidence_threshold`	`0.9`	Hallucination-probability threshold for `lettuceprevent`.
`top_k_logits`	`10`	Candidate tokens scored per step.
`last_k_tokens_to_consider`	`10`	Context window the detector uses. For 'number' 10 is usually fine, for 'lettuceprevent' use all tokens below.
`use_all_tokens`	`True`	If `False`, only the last `last_k_tokens_to_consider` tokens are used.
`model_path`	detector default	Override the hallucination-detector checkpoint.
`query`	`""`	Optional query string for certain detectors.
`debug_print`	`False`	Print per-step debug information.

Notes

Returns exactly what model.generate() would — a tensor of token ids.
The lettuceprevent detector requires a CUDA-capable GPU.
For instruction-tuned models, wrap the prompt with tok.apply_chat_template(...) (see the example above).

Citation

bibtex

@mastersthesis{Beccard:2026,
  title  = {Real-time Prevention of Factual Hallucinations in Retrieval-Augmented Generation},
  author = {Leon Beccard},
  school = {Technische Universität Wien},
  year   = {2026},
  url    = {https://repositum.tuwien.at/handle/20.500.12708/229242}
}