Welcome, Benchmarkers!

Benchmarks are at the core of Brain-Score. They are used to test how well models match experimental observations. If you want to add your experimental results into Brain-Score as a benchmark, these tutorials will guide you through the entire process.

Brain-Score supports three types of benchmarks: neural (fMRI, electrophysiology, EEG), behavioral (psychophysics, choices), and engineering (accuracy, robustness). Our tutorials cover both neural and behavioral benchmarks with detailed examples.

Tutorial Series

Follow these tutorials in order, or jump to the section most relevant to your needs:

1. What is a Benchmark?

Understanding benchmark components, interfaces, metrics, and ceilings.

Start Here →

2. Data Packaging

How to prepare StimulusSets and DataAssemblies for your benchmark.

Learn More →

3. Neural Benchmarks

Creating benchmarks for neural recording data (V1, V2, V4, IT).

Learn More →

4. Behavioral Benchmarks

Creating benchmarks for behavioral/psychophysical data.

Learn More →

5. Vision vs Language

Key differences between vision and language benchmarks.

Learn More →

6. Submitting Benchmarks

How to package, test, and submit your benchmark via GitHub PR.

Learn More →

Legacy Tutorials

These original tutorials walk through the Ferguson2024 benchmark as a detailed example:

Stuck?

Our tutorials and FAQs, created with Brain-Score users, aim to cover all bases. However, if issues arise, reach out to our community or consult the troubleshooting guide below for common errors and solutions.

Something Not Right?

If you come across any bugs, please feel free to submit an Issue on Github. One of our team members will be happy to investigate any issues.