Leaderboard Explore Compare Profile Community Blog FAQ Competition 2024 Report Issue

SeLaVi-Kinetics400

Contributed by Martin Schrimpf No reference provided Find on GitHub source Compare this model

Scores on benchmarks

from scoring API

Model rank shown below is with respect to all public models.

.001	average_vision rank 544 138 benchmarks	.001 0 ceiling best median

.002	neural_vision rank 545 77 benchmarks	.002 0 ceiling best median

.009	V1 rank 536 33 benchmarks	.009 0 ceiling best median

.073	Coggan2024_fMRI.V1-rdm v1 rank 113	.073 0 ceiling best median
	24 images Sample stimuli cannot be displayed publicly for this benchmark.

.001	V2 rank 535 10 benchmarks	.001 0 ceiling best median

.005	Coggan2024_fMRI.V2-rdm v1 rank 285	.005 0 ceiling best median
	24 images Sample stimuli cannot be displayed publicly for this benchmark.

How to use

from brainscore_vision import load_model
model = load_model("SeLaVi-Kinetics400")
model.start_task(...)
model.start_recording(...)
model.look_at(...)

Model API

Code examples

Metadata has not been added yet.

Brain Encoding Response Generator (BERG)

Through the BERG you can easily generate neural responses to images of your choice using any Brain-Score vision model.

For more information on how to use BERG, see the documentation and tutorial.

Historical trend

Hover the line for a preview; click the chart to pin that month in the sidebar. While pinned, hover does not change it -- use Release or Esc to clear.

Benchmarks BibTeX

@inproceedings{santurkar2019computer,
    title={Computer Vision with a Single (Robust) Classifier},
    author={Shibani Santurkar and Dimitris Tsipras and Brandon Tran and Andrew Ilyas and Logan Engstrom and Aleksander Madry},
    booktitle={ArXiv preprint arXiv:1906.09453},
    year={2019}
}
        @article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }
        @misc{ferguson_ngo_lee_dicarlo_schrimpf_2024,
         title={How Well is Visual Search Asymmetry predicted by a Binary-Choice, Rapid, Accuracy-based Visual-search, Oddball-detection (BRAVO) task?},
         url={osf.io/5ba3n},
         DOI={10.17605/OSF.IO/5BA3N},
         publisher={OSF},
         author={Ferguson, Michael E, Jr and Ngo, Jerry and Lee, Michael and DiCarlo, James and Schrimpf, Martin},
         year={2024},
         month={Jun}
}

metadata.yaml · Schema v2.0

This annotated template shows the structure of a model’s metadata.yaml. Each section groups related information; comments explain the expected types and allowed values.

Required: schema_version, domain, and models keyed by model identifier. Other fields may be omitted or set to null when undocumented. Use [] for empty lists.

The values below are illustrative, not this model’s metadata. Metadata changes are proposed in repository pull requests and published after merge.

# metadata.yaml — proposed schema v2.0
# Example values below are illustrative, not metadata for a real model.
# Required: schema_version, domain, and models keyed by exact model identifier.
# Other fields may be omitted or set to null when undocumented.
# Use [] for an empty list. Do not use "unknown" as a numeric/boolean value.
# Keep identifiers exact, including checkpoint suffixes.

schema_version: "2.0"  # String; quote the version.

domain: vision  # Must match the repository's registered domain.
models:
  example-model:
    sources:
      example-source:
        kind: other  # paper | huggingface | other | unreviewed
        url: https://example.org/model-card
        citation: Illustrative source, not evidence for a real model.
    model:
      display_name: Example model
      version: null  # String, e.g. "1.0".
      architecture:
        family: convolutional_neural_network  # String describing the architecture family.
        description: A convolutional model trained for image classification.
      parameter_count: 25000000  # Nonnegative integer; count, not "25M".
      parameter_count_exact: false  # Boolean; false means the count is approximate.
      trainable_layers: "All layers"  # Text; can describe partial fine-tuning.
      recurrent: false  # Boolean.
      input_resolution:
        channels: 3  # Nonnegative integer.
        height: 224  # Nonnegative integer, pixels.
        width: 224  # Nonnegative integer, pixels.
      visual_degrees: null  # Nonnegative number; vision models only.
      visual_degrees_description: null  # Text explaining the field of view.
      supervision:
        type: supervised  # String, e.g. supervised, self_supervised, weakly_supervised.
        description: Trained using class labels.

    training:
      process: null  # Text describing the training procedure.
      objective: null  # Text describing what the model learns.
      loss: null  # Text, e.g. "Cross-entropy".
      learning_rate: null  # Text; supports schedules, e.g. "0.1 with cosine decay".
      batch_size: null  # Text; supports different values across training stages.
      preprocessing: null  # Text describing normalization and augmentation.

    data:
      summary: null  # Text summarizing the data used.
      training_datasets:
        - identifier: example-training-set  # Optional dataset identifier.
          name: Example training set  # Required for each dataset entry.
          role: training  # training | pretraining | fine_tuning
          description: null  # Text describing this dataset's use.

    eval:
      test_datasets: []  # Entries: {identifier: optional, name: required, description: optional}.
      validation_datasets: []  # Same entry structure; role is implied by the list.

    io:
      modality: image  # String, e.g. image or text.
      interface: null  # Text describing the model's interface.
      input_format: null  # Text describing accepted inputs and their shape.
      output_format: null  # Text describing returned values and their shape.
      tokenizer: null  # Text; language models, where applicable.

    provenance:
      weights_provider: null  # Organization or repository providing the weights.
      checkpoint: null  # Exact checkpoint identifier.
      source_url: null  # Link to the original code or model card.
      curation_confidence: null  # Text summarizing confidence in the curated record.

    legal:
      license: null  # License identifier or full terms; distinguish code and weights.

    people:
      creators: []  # List of names, e.g. ["Example Researcher"].
      organizations: []  # List of organization names.

    use:
      applications: []  # List of intended applications (strings).
      users: []  # List of intended user groups (strings).
      limitations: []  # List of known limitations (strings).
      biases: []  # List of known biases (strings).

    lineage:
      base_models:
        - identifier: example-base  # Optional; exact identifier in the same domain.
          name: Example base model  # Required for each base-model entry.
          relationship: fine_tuned_from  # variant_of | fine_tuned_from | derived_from

    # Optional verification records. Omit this section if no verification is recorded.
    # Paths refer to fields in this YAML document, not database columns.
    assertions:
      - path: /model/architecture
        status: probable  # verified | probable | uncertain | undocumented
        sources: [example-source]  # References the structured sources above.

Region	Layer
V1	`base.layer4`
V2	`base.layer3`