Single-cell foundation-model benchmark artifacts for donor, assay, brain-region, and donor-level Alzheimer’s disease-severity shifts.
Research use only. This repository is not a diagnostic model, clinical decision-support system, medical device, or validated patient-level predictor.
Study design
What this repository is
This is a code-and-evidence release for a frozen development benchmark. It does not contain a standalone language model, C2S weights, raw SEA-AD matrices, donor-level predictions, embedding vectors, or donor-lockbox data. It packages audited reproduction code, sanitized aggregate results, frozen protocol summaries, figures, and cryptographic manifests.
The central question is whether frozen single-cell foundation-model representations provide robust or incrementally useful biological signal when evaluation respects donors as the independent unit.
Why donor-level evaluation matters
Single-cell datasets contain many cells from each donor. Treating those cells as independent replicates creates pseudoreplication and can greatly overstate certainty. NeuroCell-LM AD keeps all records from one donor together during splitting and cross-validation. The frozen design contains 56 training donors, 14 one-time development-validation donors, and 14 sealed lockbox donors.
Data, assays, regions, and cell populations
The benchmark uses official SEA-AD/Allen Institute resources acquired from upstream, not redistributed here. The broad cell-identity task covers five inhibitory subclasses—Sst, Pvalb, Vip, Lamp5, and Sncg—using MTG 10xV3.1 as the source setting, paired Multiome GEX as an assay-transfer setting, and MEC 10xV3.1 as a held-out-region setting.
The donor-level endpoint is CPS_Global, interpreted only as a cross-sectional AD-severity target. It is not longitudinal progression.
donor-level raw-count pseudobulk PCA/ridge and ElasticNet;
donor covariates and full-census cell composition;
pooled and subclass-resolved frozen C2S donor aggregates.
No C2S or Gemma weights are redistributed. The release records the pinned upstream model revision and provides scripts that recreate embeddings after users accept the upstream model and data terms.
Broad cell-subclass benchmark
Benchmark
Frozen C2S
Ranked-gene linear
Raw-HVG linear
MTG in-region
0.9921
0.9911
0.9925
MTG → Multiome
0.9911
0.9912
0.9923
MTG → MEC
0.9870
0.9843
0.9872
The broad five-subclass problem remained near saturation across donor, assay, and region shifts. Frozen C2S transferred strongly, but simpler expression-based models were practically tied. This result supports representation robustness; it does not support a material foundation-model superiority claim.
Transfer summary
Donor-level AD-severity evaluation
All preprocessing and hyperparameter selection were training-only. Seven frozen model families were then applied once to the 14 development-validation donors.
Frozen model
MAE
RMSE
Spearman ρ
R²
Frozen C2S pooled PCA/ridge
0.1689
0.2338
0.477
0.209
Covariates-only ridge
0.1785
0.2473
0.235
0.115
Raw pseudobulk PCA/ridge
0.1896
0.2555
0.376
0.055
Frozen C2S subclass-resolved PCA/ridge
0.1934
0.2579
0.512
0.038
Raw pseudobulk ElasticNet
0.2026
0.2604
0.363
0.019
Cell composition ElasticNet
0.2082
0.2657
—
-0.021
Training-mean null
0.2082
0.2657
—
-0.021
The descriptive lowest-MAE model was Frozen C2S pooled PCA/ridge (MAE 0.1689). Its point MAE improvement over covariates-only was 0.0096, with paired 95% interval [-0.0622, 0.0834]. Because the interval spans zero, the frozen incremental-value gate did not pass.
Although pooled frozen C2S had the lowest validation MAE, its mean prediction error (prediction minus observed) was 0.0628, compared with 0.0046 for the covariates-only model. Positive values indicate average overprediction. Validation calibration was prohibited and was not performed.
Validation comparison
Frozen conclusion
Supported
strong broad inhibitory-subclass transfer for frozen C2S and simpler linear representations;
donor-aware, paired benchmarking across assay and brain-region shift;
a promising descriptive point estimate for pooled frozen C2S in the small AD-severity validation cohort.
Not supported
clinical or diagnostic use;
longitudinal progression claims;
clear incremental AD-severity value beyond conventional donor covariates;
confirmatory lockbox performance;
a rationale for post-hoc QLoRA, validation calibration, or ensemble construction.
The lockbox advancement gate was not met, and the donor lockbox remained sealed.
Repository contents
scripts/research_pipeline/: approved source stages without donor identifiers;
scripts/release/: preparation, publication, remote verification, figure, and smoke-test tooling;
reports/public/: aggregate-only public results generated from frozen reports;
protocols/: sanitized frozen-method summaries;
figures/: figures generated from aggregate outputs;
manifests/: file hashes, public-release inventory, schemas, and audit results.
Reproduction
See REPRODUCIBILITY.md for the staged workflow and DATA_AVAILABILITY.md for official source acquisition. Reproducing the full analyses requires upstream SEA-AD data, substantial local disk space, and—in the C2S stages—a CUDA-capable GPU. Reviewing the public results and running the release smoke test requires only a normal Python environment.
Licensing and attribution
This is a mixed-license research release. Original release tooling and original project code are offered under Apache-2.0; original prose and figures are offered under CC BY 4.0. SEA-AD/Allen-derived aggregate evidence remains subject to the Allen Institute Terms of Use and Citation Policy, including noncommercial-use and attribution requirements. C2S and Gemma are governed by their upstream licenses. See LICENSE and THIRD_PARTY_NOTICES.md.
Limitations and nonclinical warning
See LIMITATIONS.md. The donor sample is small, the lockbox was not evaluated, technical and study confounding remain possible, and the benchmark is not a clinical validation.
