This repository contains code to identify Retrieval Transition Heads (RTH) — attention heads in multilingual large language models (LLMs) that are responsible from latent-space to target-language transition. We detect RTH via a multilingual Needle-in-a-Haystack (NIAH) paradigm: the needle (a factual statement) is placed in a haystack written in a language close to the internal… See the full description on the dataset page:
https://huggingface.co/datasets/NerdyVisky/retrieval-transition-head-cached-files.
