FEMBA is a powerful and efficient foundation model for EEG signal analysis, built upon a bidirectional Mamba state-space architecture. It supports self-supervised pre-training via masked reconstruction and end-to-end supervised fine-tuning for multiple downstream tasks (abnormal EEG, artifact detection, slowing classification). By using linear-time state-space modeling instead of quadratic attention, FEMBA scales to long EEG sequences and constrained hardware while remaining performant.
🔒 License & Usage Policy (Weights)
Weights license: The released model weights are licensed under Creative Commons Attribution–NoDerivatives 4.0 (CC BY-ND 4.0). This section summarizes the practical implications for users. This is not legal advice; please read the full license text.
✅ You may
Use and redistribute the unmodified FEMBA weights (including in commercial settings) with proper attribution to the FEMBA authors.
Fine-tune / adapt the weights for your internal use (research or production) without redistributing the modified weights.
Publish your code, configs, logs, and papers describing experiments with FEMBA (please cite the paper).
🚫 You may not
Share, host, or redistribute any modified weights (including LoRA/adapter/delta checkpoints or pruned/quantized variants). Any parameter set that encodes an adaptation is considered a derivative and cannot be shared under CC BY-ND 4.0.
Imply endorsement by the FEMBA authors for any derivative or evaluation without our written permission.
Use the FEMBA name in a way that suggests your modified model is an official FEMBA release.
🤝 How to contribute improvements (PR-gated releases)
We welcome community improvements via a pull-request (PR) workflow. If you believe your improvements should become an official FEMBA release:
Open a PR in the BioFoundation repository describing the change (architecture/head/training recipe, datasets, preprocessing, compute).
Include reproducibility artifacts: configs, seeds, scripts, environment details, training/validation logs, and the evaluation protocol (e.g., TUAB/TUAR/TUSL) with exact splits.
Provide comprehensive results (AUROC/AUPR/BA, FLOPs, memory) vs. the baselines reported in the FEMBA paper.
After maintainer review, approved changes will be retrained/validated and, if accepted, released by the maintainers as a new official FEMBA checkpoint under CC BY-ND 4.0.
Rationale: CC BY-ND protects users from fragmented, lower-quality “FEMBA variants,” while still enabling internal fine-tuning and a path for the community to upstream improvements through review.
🔎 Model Summary
Architecture: Bidirectional Mamba encoder with a 2D-conv tokenizer (patching over channels × time), random masking (60%) for SSL, and either a lightweight linear head or a Mamba-enhanced head for downstream tasks. Hidden state size is fixed at 80 across variants.
Scaling: Linear time & memory in sequence length (state-space model), enabling efficient long-context EEG modeling and on-device scenarios.
Pre-training data: >21,000 hours of unlabeled clinical EEG from Temple University Hospital (TUEG). Subjects overlapping with TUAB/TUAR/TUSL are removed to prevent leakage.
Optimization (typical): Pre-training with Smooth L1 masked-patch reconstruction; fine-tuning with Adam (LR 1e-4), cosine decay, early stopping; layer-wise LR decay factor 0.75.
🚀 Model Variants
Variant
Parameters
(num_blocks, embed_dim)
FEMBA-tiny
7.8M
(2, 35)
FEMBA-base
47.7M
(12, 35)
FEMBA-large
77.8M
(4, 79)
FEMBA-huge
386M
(20, 79)
Hidden state size is 80 for all variants; blocks correspond to Bi-Mamba layers in the encoder.
🧠 Intended Use & Limitations
Intended use. Research on EEG representation learning and downstream classification (e.g., abnormal EEG detection, artifact detection, slowing classification). FEMBA is particularly useful when long sequences or limited compute/memory preclude quadratic-cost attention.
Out-of-scope / limitations.
Not a medical device. Outputs are research signals and must not be used for clinical decision-making without appropriate validation and regulatory processes.
Domain shift. Performance can degrade across cohorts (e.g., neonatal vs. adult EEG) and label protocols; domain adaptation is encouraged.
Class imbalance. On some tasks (e.g., TUSL), AUROC may be strong while AUPR can trail attention baselines, highlighting sensitivity to class imbalance and protocol specifics.
🏗️ Architecture & Training Details
Tokenizer & patches. Raw EEG (C×T) is quartile-normalized per channel (IQR scaling) and tokenized with a 2D convolution over channel×time patches (e.g., 4×32) with learnable positional embeddings.
Self-supervised objective. Randomly mask 60% of patches; reconstruct masked content with a lightweight decoder using Smooth L1 loss (computed on masked patches only).
Encoder. Stacked Bidirectional Mamba blocks (forward + backward over a reversed copy), merged and residually connected; hidden size fixed to 80.
Fine-tuning heads.
Linear classifier: small MLP (≈0.5M params).
Mamba-enhanced classifier: adds one Mamba block before the linear layer (up to ≈0.7M params).
Optimization notes. Layer-wise LR decay (0.75); fine-tuning with Adam (initial LR 1e-4), cosine decay, early stopping; end-to-end updates (encoder + head).
📚 Training Data
Pre-training: Temple University Hospital EEG (TUEG), ~21k hours, ~15k subjects; broad clinical coverage. Overlaps with TUAB/TUAR/TUSL removed to avoid leakage.
Paper: FEMBA: Efficient and Scalable EEG Analysis with a Bidirectional Mamba Foundation Model (arXiv:2502.06438).
📜 Citation
If you use FEMBA in your research, please cite:
bibtex
1@misc{tegon2025fembaefficientscalableeeg,
2 title={FEMBA: Efficient and Scalable EEG Analysis with a Bidirectional Mamba Foundation Model},
3 author={Anna Tegon and Thorir Mar Ingolfsson and Xiaying Wang and Luca Benini and Yawei Li},
4 year={2025},
5 eprint={2502.06438},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG},
8 url={https://arxiv.org/abs/2502.06438}
9}
🛠️ Maintenance & Contact
Issues & support: please open a GitHub issue in the BioFoundation repository.
🔗 Related Models
LuMamba — Direct successor. Combines FEMBA's bi-Mamba encoder with LUNA's channel-unification cross-attention for topology-invariant EEG modeling, plus LeJEPA pre-training.
LUNA — Transformer-based topology-agnostic EEG foundation model (NeurIPS 2025). Companion architecture focused on channel-heterogeneity rather than sequence-length efficiency.
TinyMyo — Tiny foundation model for flexible EMG signal processing at the edge.
🗒️ Changelog
v1.0: Initial release of FEMBA model card with task-specific checkpoints and instructions.
