Comparison cohort

These controls define the models and benchmarks used in Compare Benchmarks.

Loading comparison data…

Wayback date Loading

Uses benchmarks and models available by the end of this UTC day.

Minimum model completedness for selected benchmarks %

Calculated from active leaves under the two selected benchmarks. Finite, non-zero scores count as complete; engineering leaves are excluded.

As ofLoading

Eligible cohort models0

NUMBER OF BENCHMARK LEAVES0

Median completednessNot available

View selected-benchmark completedness before thresholding

Benchmark scatter plot

Compare paired model scores for two benchmarks in the current comparison cohort.

Horizontal axis

Vertical axis

Hover for model details. Each dot is a model with finite scores for both selected benchmarks.

Example relationships

Select a published finding to load its benchmark pair and view the evidence.

Researchers found that the performance of computational models in classifying the object in an image (aka the popular Computer Vision ImageNet benchmark) is correlated with the alignment of the model to brain and behavioral data. This is true across different stages of the ventral stream and has been covered in several publications (e.g., Yamins & Hong et al. 2014 with a smaller image dataset, Schrimpf & Kubilius et al. 2018 with ImageNet). This connection between machine learning and neuroscience desiderata implies a striking synergy between research in these two domains, as building better-performing models might also lead to better models of the brain.

The more similar model representations are to recordings from primary visual cortex V1, the more robust the output of the model is to perturbations in the input such as image distortions and adversarial attacks ( Dapello & Marques et al. 2020). This finding is based on Brain-Score V1 benchmarks with data from Freeman & Ziemba et al. 2013 as well as ImageNet-C, and inspired the development of the VOneNet model.

As we make models more aligned to a diverse set of properties that have been discovered in primary visual cortex V1 -- such as receptive field sizes, response selectivity, surround and texture modulation, -- these more brain-aligned models also produce more human-like behavioral choices on a match-to-sample task. The V1 properties here have been compiled by Marques et al. 2021 from many classic neuroscience studies, and the behavioral data was collected by Rajalingham et al. 2018.

Correlation stability

A compact historical view of the selected relationship.

Pearson r over time; bars show the paired-model cohort.

Snapshots are quarterly before the latest year and monthly thereafter. They use the current completedness threshold and the same finite paired-score rule as the scatterplot. Select a point to update the wayback date.

Benchmark correlation matrix

Explore pairwise-complete Pearson correlations within the current comparison cohort.

Hover for Pearson r and paired n. Cells use the same scores and finite paired-model rule as the scatterplot; a coefficient requires at least eight paired models.

Model comparison

Choose two models to compare their shared benchmark scores.

Historical trend

Select two different models to see how their average_vision score and rank have evolved over time.

Per-benchmark score correlation

Each dot is a shared benchmark. Hover for details or click to visit its page; use the legend to filter domains.

Only benchmarks with finite scores for both models are shown.

Top benchmark differences

See where the selected models differ most across their shared benchmarks.

Model difference tree

See whether score differences form a coherent benchmark branch. Engineering benchmarks are excluded.

Model B higher Model A higher

Select two models with at least three shared benchmark scores.

Bar direction shows which model scores higher; length shows the difference relative to the largest visible branch difference.

Score distribution and branch rank

Compare the selected models' leaf-score distributions and their standing within each official summary branch.

Split violins use the left score axis. Smaller X markers use the reversed right rank axis, with color identifying Model A or Model B; rank 1 is best. Engineering branches are excluded.