You can deploy this model and start earning money today!

cangyeone/
seismic-snr-filtering-bias

Name: seismic-snr-filtering-bias
Availability: InStock

seismology

earthquake-monitoring

signal-to-noise-ratio

Views

No views yet

SNR-Bias GRL Reproducibility Package

This repository archives the project-controlled code, trained checkpoints, derived result tables, plotting code, and manuscript-facing figures for the GRL manuscript:

Is Signal-to-Noise Ratio a Reliable Data-Quality Criterion for Seismic Deep Learning? Curation Consequences and Monitoring Utility

The purpose of the archive is to make the manuscript results auditable without redistributing upstream datasets. Raw waveform files, continuous picker JSONL files, station databases, ambient-noise HDF5 data, CREDIT waveform HDF5, and CREDIT split-key files are not included. The archive does include the record-level indexes, manual-label metadata, derived SNR values, and seed-fixed training selections needed to audit the manuscript; these are reproducibility derivatives, not copies of the waveform dataset.

Quick Start: Open-Data Analysis Reproduction

The primary reproduction entrypoint recomputes statistics from the public waveform, continuous-pick, and dispersion data products and writes new intermediate outputs:

bash

pip install -r requirements.txt
python code/scripts/reproduce_paper_figures_from_open_data.py \
  --credit-h5 /path/to/credit-x1.h5 \
  --credit-keys /path/to/creditkeys.npz \
  --ncf-h5 /path/to/ncf_disp_dataset_with_disp_image.h5 \
  --continuous-pick-dir /path/to/SeismicX-Cont/hourly/all \
  --continuous-label-json /path/to/SeismicX-Cont/annotations_for_continuous_hdf5.json \
  --continuous-waveform-db /path/to/SeismicX-Cont/waveform_index.sqlite \
  --phase-balanced-root /path/to/phase_balanced_20190706_20211113 \
  --work-dir open_data_work

This writes the recomputed analysis outputs and training manifests under:

text

open_data_work/
  open_data_reproduction_manifest.json
  outputs/
  training_manifests/

The current manuscript compositors consume the corresponding derived tables. Superseded exploratory figure layouts can be rendered with --render-analysis-figures, but they are not the current Figures 1--3. The workflow never reads results/manuscript_figures/*_data.csv as source input. CSV/JSON files created under open_data_work/ are intermediate outputs generated during that run. The training manifests in open_data_work/ record the exact seed-fixed phase-picking and dispersion training examples selected in that run.

For detailed input expectations, smoke-test options, and the continuous association preparation commands, see OPEN_DATA_REPRODUCTION.md.

Fast Cached Figure Check

For quick visual inspection only, the archive also includes the manuscript plotted-data exports and a renderer:

python code/scripts/plot_all_paper_figures.py

This path redraws Figures 2 and 3 from cached CSV/JSON exports and packages the editable Figure 1 SVG with its values and provenance files. It is useful for checking the current manuscript figure rendering but is not the primary data-backed reproduction workflow.

The exact manuscript-facing assets in this revision are:

text

results/manuscript_figures/
  figure1_provenance_hero.pdf
  figure1_provenance_hero.svg
  figure1_provenance_hero.png
  figure1_provenance_hero_values.json
  figure1_provenance_hero_caption.txt
  figure2_training_metrics_grl.pdf
  figure2_training_metrics_grl_data.csv
  figure2_training_metrics_grl_seed_data.csv
  figure3_monitoring_metrics_grl.pdf
  figure3_monitoring_metrics_grl_data.csv

Their source scripts are archived in code/scripts/. Figure 1 uses the data-backed compositor make_grl_figure1_provenance_hero_v2.py; its plotting code and archived PDF/SVG/PNG use the final lowercase a--e panel sequence. Its provenance notes identify which elements are schematic, data-backed, or editorial.

The primary phase-picking comparison uses 39,718 records per condition, with P/S gates of 14/7.27 and 16/16.60 dB. The primary monitoring comparison uses the Vertical-S, Horizontal-S, and Probability-ranking definitions, each with 265,962 P and 265,962 S picks. Superseded stored-coordinate and common-threshold products are confined to results/legacy_exploratory/ or explicitly labeled historical sensitivities in the Supporting Information; they are not current primary conditions.

Repository Layout

text

.
├── code/
│   ├── scripts/          # Training, evaluation, aggregation, bootstrap, and plotting scripts
│   ├── odata/            # Continuous filtering and REAL-association helper scripts
│   ├── models/           # Phase-picking neural-network definitions
│   ├── utils/            # Dataset and waveform utility code used by training scripts
│   ├── dispnet.v2.3.py   # Dispersion model and training utilities
│   └── pnsn.train.v3.60s.py
├── checkpoints/
│   ├── base/             # Base PnSn v3 checkpoint used for transfer learning
│   ├── phase_picker/     # Fine-tuned and scratch phase-picking checkpoints
│   └── dispersion/       # DispNet checkpoints
├── configs/
│   └── manuscript_reproduction.json
├── training_manifests/   # Seed-fixed manifests and derived source caches
├── results/
│   ├── manuscript_figures/
│   ├── phase_picker/
│   ├── dispersion/
│   ├── multiseed/
│   ├── bootstrap/
│   └── snr_filtered_test_precision/
├── DATASETS.bib
├── OPEN_DATA_REPRODUCTION.md
├── REPRODUCIBILITY_CROSSWALK.md
├── CITATION.cff
├── LICENSE
├── NOTICE
└── CHECKSUMS.sha256

What Is Included

This archive includes:

All manuscript-facing Python scripts used for phase-picking, dispersion, filtering, aggregation, bootstrap summaries, and figure generation.
Phase-picker model definitions and the DispNet v2.3 model definition.
The base PnSn v3 checkpoint used for transfer-learning experiments.
Three-seed phase-picking checkpoints for both fine-tuning and scratch training: seed20260609, seed20260610, and seed20260611.
Three-seed dispersion checkpoints for full, medium-SNR, and high-SNR matched training.
Fixed reproduction configuration in configs/manuscript_reproduction.json, including the manuscript seeds, subset-selection seed rules, thresholds, train/evaluation budgets, and association settings.
Exact training manifests:
- training_manifests/phase_picker/seed*_train_records.jsonl.gz lists the CREDIT-X1local event/station records and manual labels selected for each seed and condition.
- training_manifests/dispersion/seed*_train_keys.txt lists the SeisDispFusion-NCF training keys selected for each seed and condition.
- training_manifests/source_caches/ stores the derived record indexes and SNR values used to regenerate those manifests; it contains no waveforms.
A condition-ID crosswalk in REPRODUCIBILITY_CROSSWALK.md. Historical file slugs are retained so archived checkpoints and result paths remain stable; they are explicitly mapped to the current P/S thresholds.
Per-seed summaries, training logs, multi-seed aggregate tables, bootstrap tables, and SNR-filtered-test precision summaries.
The open-data reproduction driver code/scripts/reproduce_paper_figures_from_open_data.py.
The direct pretrained phase-picker baseline evaluator code/scripts/evaluate_phase_direct_baseline.py.
Cached plotted-data CSV/JSON exports and the final main figures used in the current manuscript, for fast visual checks.
Documentation for matching the archived results to the GRL manuscript figures and tables.

This archive does not include:

CREDIT-X1local waveform HDF5 files or split-key files.
SeismicX-Cont continuous waveform, annotation, station-index, or pick JSONL files.
SeisDispFusion-NCF waveform/dispersion HDF5 files.
Full intermediate REAL working directories or large phase-balanced continuous pick streams.
Unreported exploratory smoke-test outputs that were not used in the GRL manuscript.

Data Sources

Download or access the data below before rerunning training, evaluation, or continuous association.

Continuous Waveform and Association Data

Dataset: SeismicX-Cont
URL: https://huggingface.co/datasets/cangyeone/SeismicX-Cont
DOI: 10.57967/hf/9006
Revision used in the manuscript: 96367f8
Used for the two-day continuous association diagnostic and labeled pick coverage checks.

Ambient-Noise Dispersion Data

Dataset: SeisDispFusion-NCF
URL: https://huggingface.co/datasets/cangyeone/SeisDispFusion-NCF
DOI: 10.57967/hf/9114
Revision used in the manuscript: afcd805
Used for the ambient-noise dispersion SNR-filtering experiment.

CREDIT-X1local

Article: CREDIT-X1local: A reference dataset for machine learning seismology from ChinArray in Southwest China
Article DOI: 10.1016/j.eqs.2024.01.018
Data-center entry: https://www.esdc.ac.cn/article/125
Dataset DOI: 10.11998/IESDC.SEIS.D03MOD20230001
Access: register with the International Earthquake Science Data Center and accept its data-sharing agreement before downloading the waveform archive.
Used for the matched phase-picking training experiment.

BibTeX entries for the data sources are in DATASETS.bib.

The primary open-data reproduction command is documented in OPEN_DATA_REPRODUCTION.md.

Environment

The scripts were run with Python 3.12 on macOS. Python 3.10 or newer should be adequate for the plotting and table scripts. Training requires PyTorch and HDF5 support.

bash

python -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt

For deterministic CPU-only checks, set:

bash

export OMP_NUM_THREADS=1
export MKL_NUM_THREADS=1
export KMP_DUPLICATE_LIB_OK=TRUE

Reproduce the Phase-Picking Experiment

Expected local inputs:

CREDIT-X1local waveform file, for example /path/to/credit-x1.h5
CREDIT split keys, for example /path/to/creditkeys.npz
Base checkpoint included here: checkpoints/base/pnsn.v3.pt

Run the three manuscript seeds. The archive includes the manuscript record indexes and phase-SNR derivatives under training_manifests/source_caches/; --cache-dir remains local for a clean rerun from the upstream waveform data:

bash

for seed in 20260609 20260610 20260611; do
  python code/scripts/snr_transfer_phase_balanced_experiment.py \
    --h5 /path/to/credit-x1.h5 \
    --keys /path/to/creditkeys.npz \
    --base-ckpt checkpoints/base/pnsn.v3.pt \
    --cache-dir local_credit_cache \
    --out-dir results/phase_picker/seed${seed}_rerun \
    --seed ${seed} \
    --train-steps 2000 \
    --scratch-train-steps 10000 \
    --train-batch 16 \
    --eval-samples 10000 \
    --init-modes finetune scratch \
    --train-phase-source MANUAL \
    --test-phase-source MANUAL \
    --p-thresholds-db 14 16 \
    --filter-mode record-any \
    --s-threshold-mode balanced \
    --match-mode phase-composition
done

For this experiment, P SNR uses 0.5 s of vertical motion after P, S SNR uses 1.5 s of horizontal-vector motion after S, and noise uses a 1.5 s window ending 2 s before P. S thresholds are calibrated separately so the numbers of threshold-passing P and S labels are equal. Training and evaluation targets use manual labels only.

The current manuscript used these matched sample budgets:

39,718 waveforms per training condition.
identical record-level phase composition: 11,575 P-only, 27,036 P+S, and 1,107 S-only records.
P/S thresholds of 14/7.272 dB and 16/16.598 dB; before final budget matching they retained 45,880 and 39,718 of 61,429 eligible records.
retained-waveform rule: a waveform is kept if any P or S label passes the phase-specific threshold, and all manual labels in that waveform are kept.

Published checkpoints from the manuscript runs are in:

text

checkpoints/phase_picker/seed20260609/
checkpoints/phase_picker/seed20260610/
checkpoints/phase_picker/seed20260611/

Each seed contains fine-tuned and scratch checkpoints for full, medium-SNR, and high-SNR matched training.

The historical archive slugs p5_s_bal and p10_s_bal are stable path identifiers, not threshold labels. They map to canonical conditions p14_s7p272 and p16_s16p598, respectively. See REPRODUCIBILITY_CROSSWALK.md; the manifest config records both names.

The exact seed-fixed training-record manifests are in:

text

training_manifests/phase_picker/
  phase_training_manifest_summary.csv
  seed20260609_full_train_records.jsonl.gz
  seed20260609_p5_s_bal_train_records.jsonl.gz
  seed20260609_p10_s_bal_train_records.jsonl.gz
  ...

Regenerate the manifests from the archived derived source caches with:

bash

python code/scripts/export_training_manifests.py \
  --out-dir training_manifests_rerun \
  --phase-records-json training_manifests/source_caches/credit_records_train_all.json.gz \
  --phase-snr-json training_manifests/source_caches/credit_train_phase_snr_pre_event_amplitude_db.json.gz \
  --phase-source MANUAL \
  --phase-p-thresholds 14 16 \
  --phase-s-threshold-mode balanced \
  --seeds 20260609 20260610 20260611

Reproduce the Dispersion Experiment

Expected local input:

SeisDispFusion-NCF HDF5 file, for example /path/to/ncf_disp_dataset_with_disp_image.h5

Run the three manuscript seeds:

bash

for seed in 20260609 20260610 20260611; do
  python code/scripts/disp_snr_transfer_experiment.py \
    --h5 /path/to/ncf_disp_dataset_with_disp_image.h5 \
    --out-dir results/dispersion/seed${seed}_rerun \
    --seed ${seed} \
    --mode scratch \
    --epochs 5 \
    --batch-size 256 \
    --device cpu \
    --num-workers 0
done

The manuscript run used:

11,033 training samples per condition.
8,292 unfiltered test samples.
training conditions: full matched, SNR >3.04 dB matched, and SNR >6.77 dB matched.
optimizer: AdamW, learning rate 2e-4.

Published trained checkpoints are in checkpoints/dispersion/seed*/.

The exact seed-fixed dispersion training-key manifests are in:

text

training_manifests/dispersion/
  dispersion_training_manifest_summary.csv
  seed20260609_full_train_keys.txt
  seed20260609_snr_q1_train_keys.txt
  seed20260609_snr_q2_train_keys.txt
  ...

Regenerate them from the archived SNR cache with:

bash

python code/scripts/export_training_manifests.py \
  --out-dir training_manifests_rerun \
  --dispersion-snr-json training_manifests/source_caches/ncf_snr_cache_seed20260609.json.gz \
  --seeds 20260609 20260610 20260611

Reproduce the SNR-Filtered Test Precision Table

After the phase-picking checkpoints and CREDIT-X1local inputs are available, rerun the Table 2 precision/recall audit with:

bash

python code/scripts/snr_filtered_test_precision_table.py \
  --h5 /path/to/credit-x1.h5 \
  --keys /path/to/creditkeys.npz \
  --cache-dir local_credit_cache \
  --test-phase-source MANUAL \
  --test-filter-mode record-any \
  --eval-cache-tag manual_pre_event_record_any \
  --source-dirs \
    results/phase_picker/seed20260609_rerun \
    results/phase_picker/seed20260610_rerun \
    results/phase_picker/seed20260611_rerun \
  --out-dir results/snr_filtered_test_precision_rerun

The manuscript-facing saved results are in results/snr_filtered_test_precision/.

Aggregate the Three Training Seeds

The saved manuscript aggregates are in results/multiseed/. To recompute aggregates from a parent-project layout, run:

bash

python code/scripts/grl_aggregate_multiseed.py \
  --seeds 20260609 20260610 20260611 \
  --phase-balanced \
  --phase-prefix snr_transfer_manual_pre_event_seed \
  --out-dir results/multiseed_rerun

If running directly inside this archive after retraining, point the script or copy the rerun summaries so the expected outputs/<experiment>/summary.json layout is available.

Continuous Association Experiment

The continuous analysis is a two-day definition-comparison experiment using the same learned picker output stream but different retention rules before REAL association. The thresholds were calibrated and evaluated on these same dates, so this archive does not describe the result as independent monitoring validation.

Manuscript settings:

days: 2019-07-06 and 2021-11-13
SNR conversion: 20*log10(std_post/std_pre) on the amplitude-dB scale
Vertical-S definition: P and S use vertical-component standard deviation in adjacent 2-s windows; thresholds are P 10.00 dB, S 7.451 dB
Horizontal-S definition: P uses the vertical component and S uses the horizontal-vector standard deviation; thresholds are P 10.00 dB, S 8.142 dB
Probability-ranking definition: P and S are ranked separately by picker probability
retained phase budget: 265,962 P + 265,962 S in each stream
REAL chunking: 15-min windows
REAL -R: 0.4/25/0.05/3/5
REAL -S: 4/2/3/2/1.0/0.1/1.0
event matching: 5 s origin time and 30 km epicentral distance, with 2 s/10 km and 3 s/20 km robustness checks
temporal-budget sensitivity: Probability-ranking picks are also matched by phase separately inside each 15-min association window
component sensitivity: Vertical-S and Horizontal-S rankings are also compared over the same three-component-eligible candidate IDs

Helper scripts are in code/odata/. The saved manuscript summaries are in results/continuous_association/, results/bootstrap/, and results/manuscript_figures/.

Main Numerical Results in the Archive

Fixed-budget continuous association:

Definition	P picks	S picks	Event TP	Event precision	Event recall	Event F1
Vertical-S definition	265,962	265,962	1,255	0.651	0.536	0.588
Horizontal-S definition	265,962	265,962	1,504	0.644	0.643	0.643
Probability ranking	265,962	265,962	1,357	0.509	0.580	0.542

Paired Horizontal-S-minus-Vertical-S block bootstrap:

S-arrival recall difference: +0.324, 95% interval [0.314, 0.335]
catalog-event recall difference: +0.106, 95% interval [0.091, 0.123]
catalog-event F1 difference: +0.055, 95% interval [0.030, 0.079]
catalog-referenced precision difference: -0.007, 95% interval [-0.035, 0.027]
reference catalog events: 2,340
resamples: 10,000

Reference-arrival recall at the same global phase budget:

Definition	P recall	S recall
Vertical-S definition	0.691	0.394
Horizontal-S definition	0.691	0.718
Probability ranking	0.615	0.441

Common-eligibility component comparison:

Definition	Eligible S candidate IDs	P recall	S recall	Event precision	Event recall	Event F1
Vertical-S definition	1,292,360	0.691	0.473	0.674	0.571	0.618
Horizontal-S definition	1,292,360	0.691	0.662	0.650	0.645	0.648

Both rows used identical P picks and matched S counts in every 15-minute window. The audit and metrics are in results/continuous_association/phase_aware_primary_20260723/common_eligibility_component/.

Under retrospective 15-min phase-budget matching, Horizontal-S/Probability-ranking event recall was 0.645/0.666, catalog-referenced precision was 0.650/0.531, and F1 was 0.647/0.591. Compact definition-comparison and sensitivity outputs are in results/continuous_association/phase_aware_primary_20260723/; the raw continuous waveforms and large intermediate pick streams remain in SeismicX-Cont.

Matched learning results are stored in:

text

results/multiseed/phase_any/
results/multiseed/phase_dispersion/
results/snr_filtered_test_precision/

For phase picking using manual labels, the higher training gate retained 64.7% of eligible records before budget matching. At 39,718 records per condition, transfer P/S F1 on unfiltered tests was 0.817/0.618 for full-distribution training and 0.813/0.608 for the higher gate. Scratch P/S F1 was 0.773/0.563 and 0.761/0.535, respectively. Cross-evaluation found no reversal on the corresponding gated test set; full-distribution training was equal or better within rounding. These values support data-volume and geometry changes without claiming a universal performance loss from every SNR subset.

Check File Integrity

After download, verify the archive contents:

bash

shasum -a 256 -c CHECKSUMS.sha256
python code/scripts/verify_repository_manuscript_alignment.py

If you rerun scripts, generated files will change and the checksum file should be regenerated for the new archive state.

Reproducibility Boundaries

This archive supports two levels of reproducibility.

First, manuscript figures and tables can be checked from the archived derived CSV/JSON summaries and plotted-data exports without downloading the raw data. This is the expected quick rendering check for reviewers.

Second, full training, testing, SNR filtering, and association can be rerun after downloading the public datasets listed above. These reruns require substantial storage and compute. The intended entrypoint is code/scripts/reproduce_paper_figures_from_open_data.py; the local data paths are explicit command-line arguments because raw data are not redistributed here.

The archive does not redistribute the manuscript source, manuscript PDF, or Supporting Information. The submitted GRL manuscript cites this repository URL and an immutable commit hash in the Open Research section. This avoids a circular manuscript snapshot whose embedded archive revision would become stale when the repository is updated.

The archive includes fixed seeds, exact phase and dispersion training manifests, and the derived CREDIT record-index, manual-label, and phase-SNR caches needed to audit their construction. It does not redistribute raw CREDIT-X1local, SeismicX-Cont, or SeisDispFusion-NCF waveform/dispersion arrays. Those raw inputs must be obtained from the cited sources.

The archive does not claim that the reported SNR thresholds are universal physical thresholds across tasks. Phase-picking SNR and dispersion SNR are task-specific filtering scales, as described in the manuscript and SI.

Citation

Please cite this archive, the GRL manuscript, and the upstream datasets when reusing the materials. A CITATION.cff file and data-source BibTeX entries are included. The repository URL for this archive is:

https://huggingface.co/cangyeone/seismic-snr-filtering-bias

The DOI for this archive is 10.57967/hf/9115.

Each repository correction is preserved as a new immutable Git revision. The manuscript and Supporting Information are maintained through the journal submission workflow rather than duplicated in this archive.

License

Unless otherwise noted, project-controlled code, derived outputs, summary tables, figure assets, model checkpoints, and reproducibility notes in this archive are released under the Creative Commons Attribution 4.0 International license (CC BY 4.0).

Raw datasets are not redistributed in this archive and should be used under the terms stated by their original repositories or publications. Included CREDIT-derived indexes, manual-label metadata, SNR values, and selection manifests retain the provenance and applicable usage terms of the cited upstream source.