This model is a Transformer-based encoder for heart rate (HR) sequences, designed to learn robust representations of short-term HR fluctuations (HF components) in a self-supervised pretraining setup.
Key features:
- HR-only input: Sequence of heart rate measurements (BPM).
- Adaptive Normalization: Internal Normalization layer learns mean and variance from training HR data.
- Pre-LN Transformer: Multi-layer Pre-LayerNorm Transformer with residual connections for stable sequence modeling.
- Masked pretraining: Randomly masks portions of the HR sequence during training to learn contextual representations.
- Robust to short-term HR spikes: Designed to handle physiological or situational changes (e.g., exercise, stress, sudden excitement).
Intended Use
- Pretraining for downstream HR/HRV tasks, such as:
- Heart rate prediction / imputation
- Wearable biosignal modeling
- Works on fixed-length HR windows, e.g., 128-minute sequences.
Training Data
- Derived from the AFDB (Atrial Fibrillation Database) ECG recordings.
- HR sequences extracted via fast R-peak detection and sliding-window HR computation.
- Masking ratio: 0.05 (configurable during training).
Evaluation Metrics
- Masked sequence reconstruction evaluated via:
- Mean Absolute Error (MAE)
- Root Mean Squared Error (RMSE)
- R² (variance explained)
AFDB Citation
Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P. C., Mark, R., ... & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215–e220. RRID:SCR_007345.
