Symbol	Description
$ \text{score}_{ij} $	Attention score between query at position $ i $ and key at position $ j $
$ Q_i $	Query vector for position $ i $
$ K_j $	Key vector for position $ j $
$ d_k $	Dimension of key vectors
$ \gamma $	Learnable time bias strength
$ f(\cdot) $	Time difference function
$ t_j - t_i $	Relative time difference

How To Use

python

from temporal_attention import TemporalSelfAttention

model = TemporalSelfAttention(
    embed_dim=64,
    num_heads=1,
    bias_type="linear",  # or 'gaussian'
    gamma=1.0,
    causal=False
)

# x: (B, T, D), timestamps: (B, T)
output, weights = model(x, timestamps)

Symbol	Description
\( \text{score}_{ij} \)	Attention score between query at position \( i \) and key at position \( j \)
\( Q_i \)	Query vector for position \( i \)
\( K_j \)	Key vector for position \( j \)
\( d_k \)	Dimension of key vectors
\( \gamma \)	Learnable time bias strength
\( f(\cdot) \)	Time difference function
\( t_j - t_i \)	Relative time difference

sanskxr02/temporal-self-attention

TemporalSelfAttention - A Time-Biased Attention Module

Why?

Core Equation

Notation

How To Use

sanskxr02/
temporal-self-attention