This repository contains the Fovea-Gaze SimCLR ResNet-18 checkpoint from:
Diaz, D. M., & Henderson, M. M. (2026). Eccentricity-Constrained CNN Training Reveals Adaptive Information Coding Around the Visual Field. Proceedings of the 9th Conference on Cognitive Computational Neuroscience.
This model is part of the Eccentricity-Constrained SimCLR Models (VEDB)collection, containing checkpoints pretrained under four visual-field conditions: Baseline, Fovea-Gaze, Periph, and Periph-NF.
This model uses a ResNet-18 visual encoder pretrained with SimCLR self-supervised contrastive learning on naturalistic egocentric imagery with synchronized human gaze data from the Visual Experience Dataset (VEDB).
The associated study investigated whether constraining visual experience to different portions of the visual field produces systematic differences in learned representations, downstream task performance, and alignment with human visual cortex.
Fovea-Gaze Condition
Example VEDB frames under the Baseline, Fovea-Gaze, Periph, and Periph-NF training conditions
The Fovea-Gaze condition isolates gaze-centered central visual information. For each 224 × 224 VEDB frame, a fixed 112 × 112 crop was extracted around the participant's per-frame gaze location. Crop coordinates were clamped to the image boundaries and the resulting crop was upsampled to 224 × 224 using bilinear interpolation to preserve a common network input size across conditions.
A feathered circular aperture was applied to suppress information outside the gaze-centered crop, with the surrounding region masked using uniform gray (128) and Gaussian-blurred aperture boundaries (kernel = 15). When a reliable gaze location was unavailable, the image center was used as a fallback fixation location.
The Fovea-Gaze condition should therefore be interpreted as a gaze-centered central-only input condition, rather than as a biological simulation of the human fovea.
The associated model variants manipulate the same source frames in complementary ways:
Baseline: full-field input without an eccentricity-specific restriction
Fovea-Gaze: gaze-centered central-only input
Periph: peripheral-only input produced by masking the corresponding gaze-centered central region
Periph-NF: peripheral-only input with a NeuroFovea transform applied before central masking
Full dataset not redistributed; example frames included for illustration; see Databrary
Technical Provenance Note and Discrepancies
Technical provenance note: The checkpoint metadata, architecture, training parameters, and implementation details documented in this model card have been re-verified against the released model checkpoint and, where available, the original training code and launch configuration. If a technical detail concerning the released model artifact differs between the associated paper and this model card, the model card should be treated as the authoritative description of the released checkpoint and its implementation. The associated paper remains the primary source for the study's scientific analyses, results, and interpretation.
Checkpoint
File:
simclr_resnet18_fovea-gaze_epoch120.pth.tar
This repository provides the original PyTorch training checkpoint from epoch 120 of Fovea-Gaze SimCLR pretraining.
The checkpoint is serialized as a dictionary containing:
Key
Contents
epoch
Final training epoch (120)
arch
Backbone architecture (resnet18)
state_dict
Model parameters and registered buffers
optimizer
Adam optimizer state at the time of saving
The state_dict contains 124 entries and includes both the ResNet-18 encoder and SimCLR projection head.
Model parameters use the backbone.* namespace. The projection head is stored as:
backbone.fc.0: Linear(512, 512)
backbone.fc.2: Linear(512, 128)
The checkpoint therefore contains the full SimCLR model state rather than encoder weights alone.
Architecture
Architecture overview
Overview of the VEDB preprocessing, SimCLR pretraining, downstream linear probes, and voxelwise encoding workflow.
Component
Specification
Backbone
ResNet-18
Framework
PyTorch
Learning paradigm
Self-supervised contrastive learning
Objective
SimCLR / NT-Xent
Input resolution
224 × 224
Backbone representation
512-dimensional
Projection head
Linear(512, 512) → ReLU → Linear(512, 128)
Projection dimension
128
The standard ResNet-18 classification layer was replaced during SimCLR pretraining by a two-layer projection head:
The model was pretrained using imagery from the Visual Experience Dataset (VEDB), a large-scale dataset of naturalistic egocentric experience containing more than 200 hours of integrated egocentric video, eye-movement, and odometry recordings.
The full VEDB dataset is not redistributed through this repository. A small number of example frames are included solely to illustrate the visual-field transformations used in the study. Researchers wishing to reproduce training should obtain VEDB through the official distribution and comply with its applicable access and usage requirements.
Dataset Construction
A total of 717 VEDB sessions were initially retrieved through Databrary. Sessions without synchronized gaze data were excluded, leaving 514 sessions for processing and analysis.
Within task-relevant portions of each retained session:
Frames were sampled every 2 seconds.
Sampling used the native 25 FPS video rate.
A maximum of 1,000 frames per session was sampled.
The resulting SimCLR dataset contained 433,564 frames.
All data splits were performed at the video-session level to prevent leakage from temporally adjacent and environmentally correlated frames.
SimCLR Dataset Split
Split
Sessions
Frames
Frame proportion
Train
455
377,462
87.06%
Validation
28
26,026
6.00%
Test
31
30,076
6.94%
Total
514
433,564
100%
The split was constructed as an approximately 80/10/10 session-level split using stratification to balance task-label representation. Because sessions contain different numbers of sampled frames, the resulting frame percentages differ from the session-level proportions.
Validation and test sessions were held out from all representation learning and hyperparameter selection.
Frame Preprocessing
Sampled VEDB frames were first processed using a deterministic common pipeline:
Decode the sampled video frame.
Convert the image to RGB.
Bicubic resize to 256 px.
Center crop to 224 × 224.
For the Fovea-Gaze condition, the following condition-specific transformation was then applied:
Obtain the participant's synchronized per-frame gaze location.
Extract a fixed 112 × 112 gaze-centered crop from the 224 × 224 frame.
Clamp the crop coordinates to the image boundaries when necessary.
Upsample the crop to 224 × 224 using bilinear interpolation.
Apply a feathered circular aperture with a uniform gray surround (128) and Gaussian-blurred aperture boundary (kernel = 15).
Use the image center as a fallback fixation location when reliable gaze information is unavailable.
This condition-specific construction occurred before the shared SimCLR augmentation pipeline.
Because the crop is upsampled from 112 × 112 to 224 × 224, the Fovea-Gaze condition isolates central image content while also changing the effective spatial scale and interpolation characteristics of the input relative to the full-field Baseline condition.
SimCLR Pretraining
The four VEDB conditions used the same SimCLR architecture (PyTorch-SimCLR), optimization procedure, and augmentation pipeline. They differed only in the visual-field manipulation applied to the source imagery before SimCLR augmentation.
Hyperparameter
Value
Backbone
ResNet-18
Input size
224 × 224
Epochs
120
Batch size
512
Optimizer
Adam
Learning rate
6 × 10^-4
Weight decay
1 × 10^-4
Loss
NT-Xent
Temperature (τ)
0.07
Learning-rate schedule
CosineAnnealingLR (scheduler stepping begins after epoch 10)
Projection head
Linear(512,512) → ReLU → Linear(512,128)
Mixed precision
FP16
SimCLR Augmentations
The common SimCLR augmentation pipeline included:
random resized cropping,
random horizontal flipping,
color jitter (p = 0.8),
grayscale conversion (p = 0.2), and
Gaussian blur with σ ~ U(0.1, 2.0).
The same augmentation pipeline was used across all four VEDB conditions and was applied after condition-specific frame construction. See PyTorch-SimCLR for further SimCLR implementation details.
Evaluation
Following SimCLR pretraining, the frozen ResNet-18 backbone was evaluated using linear probes for in-domain and out-of-domain classification and voxelwise encoding models for neural prediction.
Comparative Evaluation Results
The table below reproduces the summary metrics reported in the associated paper across all VEDB-trained conditions and reference models. Rows corresponding to this repository's Fovea-Gaze checkpoint are bolded.
Task
Condition
Val Loss
Top-1 (%)
Top-5 (%)
Best Macro-F1 (%)
SimCLR
Baseline
0.4331
87.60
—
—
SimCLR
Fovea-Gaze
0.3749
90.43
—
—
SimCLR
Periph-NF
0.4548
90.04
—
—
SimCLR
Periph
0.4545
89.26
—
—
In-Domain
Baseline
0.9811
—
—
42.17
In-Domain
Fovea-Gaze
1.2031
—
—
43.64
In-Domain
Periph-NF
1.3090
—
—
30.93
In-Domain
Periph
1.0623
—
—
36.56
In-Domain
STL-10
1.6666
—
—
25.41
In-Domain
ImageNet-100
1.2342
—
—
41.23
In-Domain
ImageNet-1K
0.9713
—
—
43.33
VGGFace2
Baseline
7.8101
5.21
11.73
3.26
VGGFace2
Fovea-Gaze
7.9104
4.58
10.76
2.70
VGGFace2
Periph-NF
8.0232
3.39
8.17
1.90
VGGFace2
Periph
8.1681
2.54
6.39
1.35
VGGFace2
STL-10
6.9973
9.55
18.96
7.43
VGGFace2
ImageNet-100
6.7985
10.77
21.07
8.71
VGGFace2
ImageNet-1K
6.7964
10.74
21.08
8.77
Places365
Baseline
3.9690
25.63
51.90
23.16
Places365
Fovea-Gaze
4.2347
21.86
46.21
19.14
Places365
Periph-NF
4.2621
20.51
44.58
17.86
Places365
Periph
4.2671
20.26
44.10
17.65
Places365
STL-10
3.8281
26.57
53.47
24.82
Places365
ImageNet-100
3.9207
24.99
51.21
23.32
Places365
ImageNet-1K
3.6264
30.17
58.46
28.36
Note: SimCLR Top-1 is computed from the self-supervised contrastive objective and is not directly comparable to downstream supervised classification accuracy. For downstream tasks, the pretrained ResNet-18 backbone was frozen and only a linear classifier was trained; the backbone weights were not fine-tuned. Classifier checkpoints were selected by best validation Macro-F1. In-domain Top-1 accuracy is omitted because label imbalance across frames can make accuracy misleading; Macro-F1 is reported as the primary class-balanced metric. STL-10, ImageNet-100, and ImageNet-1K are treated as out-of-domain baselines because they were not pretrained on VEDB.
For in-domain classification, Macro-F1 was used as the primary class-balanced metric because of label imbalance across VEDB frame categories.
Neural Encoding Evaluation
The pretrained model was additionally evaluated using voxelwise encoding models of human fMRI responses from the Natural Scenes Dataset (NSD).
NSD contains 7T whole-brain fMRI responses to complex natural scenes. The analysis used data from 8 human participants.
For each model:
NSD stimuli were provided as intact 224 × 224 images; the VEDB visual-field transformations used during SimCLR pretraining were not reapplied at the encoding stage,
images were rescaled to [0, 1] and normalized using ImageNet channel means and standard deviations,
features were extracted from Conv1, Layer1.1, Layer2.1, Layer3.1, Layer4.1, and Avgpool,
convolutional feature maps were spatially reduced using adaptive average pooling and flattened,
each layer's features were dimensionally reduced to the top 200 principal components,
the resulting layer-wise feature sets were concatenated and z-scored,
voxelwise L2-regularized linear regression (ridge) models were fit, with the regularization parameter selected independently for each voxel using a nested holdout partition.
For each participant, the 1,000 NSD images shared across all participants served as the final held-out evaluation set. The remaining subject-specific images were divided into a training partition and a nested holdout partition used for ridge-parameter selection.
Encoding performance was quantified using held-out voxelwise R².
Importantly, original intact NSD images were presented to every pretrained model during encoding evaluation. The Baseline, Fovea-Gaze, Periph, and Periph-NF visual-field transformations were applied during SimCLR pretraining and were not reapplied to NSD stimuli at the encoding stage.
For complete ROI-level prediction accuracy, statistical comparisons, variance-partitioning analyses, and comparisons across eccentricity conditions, see the associated paper.
Intended Use
This checkpoint is provided primarily for research involving:
self-supervised visual representation learning,
egocentric visual experience,
gaze-contingent visual processing,
central versus peripheral information processing,
visual-field eccentricity,
transfer learning and linear probing,
computational modeling of visual cortex, and
model-to-brain comparisons.
The checkpoint may also be used as a pretrained ResNet-18 initialization for methodological extensions or comparisons with alternative gaze-centered or visual-field manipulations.
Out-of-Scope Use
This model was developed as a research representation-learning model and was not designed or validated as:
a production image-classification system,
a general-purpose computer-vision foundation model,
a biological simulation of the human visual system or human fovea, or
a system for making decisions about individuals.
The Fovea-Gaze manipulation isolates gaze-centered central information using a fixed crop and aperture. It should not be interpreted as a physiological model of retinal sampling, acuity falloff, cortical magnification, or other biological properties of foveal vision.
Limitations
VEDB consists of naturalistic first-person visual experience and is consequently more temporally correlated and semantically constrained than large curated computer-vision datasets.
Only one SimCLR pretraining run per VEDB condition was used in the published study. These checkpoints therefore do not characterize variation across independent pretraining seeds.
The Fovea-Gaze condition uses a fixed 112 × 112 gaze-centered crop that is bilinearly upsampled to 224 × 224. Accordingly, differences between Fovea-Gaze and the other conditions may reflect both the information available within the gaze-centered region and consequences of cropping, resizing, and interpolation.
The learned representations are specific to the architecture, training objective, augmentations, data-sampling procedure, visual-field manipulation, and preprocessing choices used in the study. Alternative implementations may produce different representations or downstream performance.
These weights should therefore be interpreted as reproducible artifacts of the published experimental conditions. The authors do not make the claim that the particular implementation is the uniquely optimal method for modeling visual-field eccentricity.
Loading the Model
The checkpoint contains the complete SimCLR model state, including the ResNet-18 encoder and projection head.
To use the pretrained ResNet-18 representation without the SimCLR projection head:
python
1# x should be a preprocessed image tensor with shape [B, 3, 224, 224]23encoder = model.backbone
4encoder.fc = nn.Identity()56with torch.no_grad():7 features = encoder(x)89print(features.shape)10# torch.Size([1, 512])
SimCLR ResNet-18 — STL-10(external pretrained reference model; checkpoint provided by Spijkervet/SimCLR and not redistributed by this project)
Computational Resources
Model training and computational analyses for this study were conducted primarily using Carnegie Mellon University Neuroscience Institute's MiND computing cluster.
Citation
If you use these model weights in academic work, please cite the associated study:
bibtex
1@inproceedings{diaz2026eccentricity,
2 author = {Diaz, Dylan M. and Henderson, Margaret M.},
3 title = {Eccentricity-Constrained CNN Training Reveals Adaptive Information Coding Around the Visual Field},
4 booktitle = {Proceedings of the 9th Conference on Cognitive Computational Neuroscience},
5 address = {New York, NY, USA},
6 year = {2026},
7 doi = {10.32470/0416gfsq}
8}
Proceedings: Conference on Cognitive Computational Neuroscience 2026 Preprint:arXiv:2607.19316
VEDB Citation
Researchers using the underlying Visual Experience Dataset should also cite:
Greene, M. R., et al. (2024). The Visual Experience Dataset: Over 200 recorded hours of integrated eye movement, odometry, and egocentric video. Journal of Vision, 24(11), 6.
See the VEDB dataset paper for the complete author list and citation information.
License
The model checkpoint in this repository is released under the Apache License 2.0.
The VEDB dataset and other third-party resources used in the associated study remain subject to their respective licenses, access requirements, and terms of use. This repository does not redistribute the full VEDB dataset; a small number of example frames are included for illustration of the published visual-field manipulations.
