Score Legend No Alignment Alignment at ceiling

Rank	Model	average 3	neural 2	Pereira2018-linear 2	Pereira2018.384sentences-linear v1	Pereira2018.243sentences-linear v1	behavior 1	Futrell2018-pearsonr v1	engineering 30	SyntaxGym 30	syntaxgym-number_src v1	syntaxgym-npi_src_any v1	syntaxgym-subordination_src-src v1	syntaxgym-reflexive_src_masc v1	syntaxgym-subordination v1	syntaxgym-subordination_orc-orc v1	syntaxgym-subordination_pp-pp v1	syntaxgym-center_embed v1	syntaxgym-cleft v1	syntaxgym-fgd_hierarchy v1	syntaxgym-mvrr v1	syntaxgym-fgd_subject v1	syntaxgym-fgd_object v1	syntaxgym-reflexive_prep_fem v1	syntaxgym-npz_ambig v1	syntaxgym-fgd_pp v1	syntaxgym-center_embed_mod v1	syntaxgym-npz_obj_mod v1	syntaxgym-mvrr_mod v1	syntaxgym-npz_ambig_mod v1	syntaxgym-cleft_modifier v1	syntaxgym-reflexive_prep_masc v1	syntaxgym-reflexive_src_fem v1	syntaxgym-npi_orc_ever v1	syntaxgym-npi_orc_any v1	syntaxgym-number_orc v1	syntaxgym-reflexive_orc_masc v1	syntaxgym-number_prep v1	syntaxgym-reflexive_orc_fem v1	syntaxgym-npi_src_ever v1
1	gpt-neo-1.3B Brain-Score Team	.662	1.0	1.0	1.0	1.0	.324	.324	.800	.800	.737	.974	1.0	.632	1.0	1.0	1.0	.929	1.0	.000	.750	.542	.958	.368	.958	.792	.893	1.0	.714	1.0	1.0	.789	.263	1.0	.974	.737	.947	.789	.263	1.0
2	gpt2-xl Brain-Score Team	.652	.987	.987	1.0	.973	.318	.318	.825	.825	.789	.974	1.0	.737	1.0	1.0	1.0	1.0	1.0	.000	.714	.458	.958	.579	.792	.833	.929	1.0	.893	1.0	1.0	.947	.421	1.0	1.0	.737	.789	.789	.474	.947
3	gpt-neo-2.7B Brain-Score Team	.649	.990	.990	1.0	.980	.308	.308	.847	.847	.842	.974	1.0	.842	.957	1.0	1.0	.964	1.0	.000	.786	.458	.917	.474	.958	.833	.893	1.0	.786	1.0	1.0	.842	.579	1.0	1.0	.789	1.0	.895	.632	1.0
4	gpt2-large Brain-Score Team	.639	.948	.948	.974	.922	.331	.331	.786	.786	.789	.974	.957	.474	.957	.957	.957	.964	1.0	.000	.821	.458	.958	.474	.583	.958	.929	.958	.857	.875	1.0	.737	.368	1.0	1.0	.632	.789	.842	.316	1.0
5	gpt2-medium Brain-Score Team	.634	.918	.918	.927	.908	.350	.350	.796	.796	.789	1.0	1.0	.684	.957	1.0	1.0	.964	1.0	.000	.750	.625	.958	.579	.583	.958	.893	.792	.821	.792	1.0	.737	.421	1.0	1.0	.579	.789	.842	.368	1.0
6	gpt2 Brain-Score Team	.608	.850	.850	.828	.871	.367	.367	.762	.762	.789	.579	1.0	.632	1.0	1.0	1.0	.929	1.0	.000	.786	.500	.958	.211	.667	.875	.857	.958	.750	.875	.925	.789	.158	1.0	.974	.632	.842	.789	.474	.921
7	openai-gpt Brain-Score Team	.594	.802	.802	.800	.804	.387	.387	.523	.523	.263	.263	.478	.105	.217	.478	.391	1.0	1.0	.000	.893	.333	.958	.474	.917	.792	.964	.958	.857	.917	.975	.474	.158	.368	.289	.105	.105	.421	.105	.421
8	distilgpt2 Brain-Score Team	.590	.818	.818	.815	.822	.361	.361	.503	.503	.789	.000	.565	.526	.217	.957	.478	.964	1.0	.000	.821	.542	.875	.105	.667	.875	.929	.875	.786	.667	.725	.474	.158	.026	.026	.105	.368	.579	.000	.000
9	gpt-neo-125m	.571	.784	.784	.788	.781	.358	.358	.736	.736	.737	.737	.913	.632	.957	.826	.957	.964	1.0	.000	.786	.625	1.0	.421	.833	.917	.857	1.0	.786	.917	.925	.579	.263	1.0	.947	.421	.526	.579	.105	.868
10	lm1b Brain-Score Team	.433	.580	.580	.568	.593	.286	.286	.408	.408	.684	.921	.913	.474	.913	.870	.957	.786	X	X	X	X	X	.632	X	X	.607	X	X	X	X	.632	.211	.158	.711	.526	.632	.737	.368	.500
11	randomembedding-1600 Brain-Score Team	.166	.332	.332	.419	.244	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
12	glove-840b Brain-Score Team	.107	.213	.213	.184	.242	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X		X		X	X	X	X	X	X	X	X	X
13	randomembedding-100 Brain-Score Team	.062	.125	.125	.221	.029	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
14	earley-parser-minivocab Brain-Score Team	X	X	X	X	X	X	X	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000	.000

How to Interpret

The leaderboard is the heart of Brain-Score. It displays scores, all ranging from 0-1 (where 1 indicates closest alignment between model and collected data), on every benchmark available. Benchmarks are arranged hierarchically, with scores for each child benchmark averaged together to create the overall score for the parent.

Ready to Submit

If you would like to submit your own model to Brain-Score, we highly recommend you complete our tutorial series here. It covers how to score a model on a single benchmark locally, walkthroughs of model submission packages, and more.

Welcome to Brain-Score!

Leaderboard

How to Interpret