Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Baker2022fragmented-accuracy_delta")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.987
2
.987
3
.986
4
.986
5
.985
6
.984
7
.984
8
.984
9
.983
10
.982
11
.982
12
.982
13
.981
14
.978
15
.975
16
.970
17
.965
18
.964
19
.960
20
.960
21
.960
22
.957
23
.954
24
.946
25
.945
26
.944
27
.944
28
.937
29
.935
30
.926
31
.925
32
.917
33
.909
34
.903
35
.901
36
.901
37
.901
38
.889
39
.882
40
.881
41
.868
42
.860
43
.858
44
.858
45
.838
46
.836
47
.836
48
.834
49
.832
50
.832
51
.822
52
.811
53
.806
54
.803
55
.802
56
.799
57
.796
58
.791
59
.788
60
.787
61
.785
62
.774
63
.760
64
.758
65
.756
66
.755
67
.751
68
.740
69
.739
70
.735
71
.734
72
.734
73
.730
74
.723
75
.721
76
.720
77
.709
78
.700
79
.698
80
.691
81
.684
82
.671
83
.670
84
.663
85
.656
86
.649
87
.646
88
.626
89
.617
90
.603
91
.602
92
.597
93
.592
94
.590
95
.583
96
.582
97
.575
98
.566
99
.558
100
.558
101
.551
102
.550
103
.543
104
.541
105
.538
106
.532
107
.528
108
.524
109
.523
110
.515
111
.507
112
.499
113
.494
114
.478
115
.473
116
.470
117
.446
118
.445
119
.438
120
.433
121
.424
122
.421
123
.417
124
.412
125
.412
126
.412
127
.411
128
.400
129
.392
130
.392
131
.388
132
.365
133
.350
134
.336
135
.336
136
.333
137
.323
138
.308
139
.308
140
.304
141
.289
142
.287
143
.282
144
.280
145
.274
146
.272
147
.268
148
.264
149
.251
150
.236
151
.221
152
.217
153
.216
154
.204
155
.195
156
.195
157
.186
158
.178
159
.167
160
.166
161
.161
162
.149
163
.124
164
.115
165
.111
166
.096
167
.096
168
.053
169
.038
170
.032
171
.030
172
.029
173
.021
174
.015
175
.014
176
.011
177
.011
178
.003
179
.000
180
.000
181
.000
182
.000
183
.000
184
.000
185
.000
186
.000
187
.000
188
.000
189
.000
190
.000
191
.000
192
.000
193
.000
194
.000
195
.000
196
.000
197
.000
198
.000
199
.000
200
.000
201
.000
202
.000
203
.000
204
.000
205
.000
206
.000
207
.000
208
.000
209
.000
210
.000
211
.000
212
.000
213
.000
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311

Benchmark bibtex

@article{BAKER2022104913,
                title = {Deep learning models fail to capture the configural nature of human shape perception},
                journal = {iScience},
                volume = {25},
                number = {9},
                pages = {104913},
                year = {2022},
                issn = {2589-0042},
                doi = {https://doi.org/10.1016/j.isci.2022.104913},
                url = {https://www.sciencedirect.com/science/article/pii/S2589004222011853},
                author = {Nicholas Baker and James H. Elder},
                keywords = {Biological sciences, Neuroscience, Sensory neuroscience},
                abstract = {Summary
                A hallmark of human object perception is sensitivity to the holistic configuration of the local shape features of an object. Deep convolutional neural networks (DCNNs) are currently the dominant models for object recognition processing in the visual cortex, but do they capture this configural sensitivity? To answer this question, we employed a dataset of animal silhouettes and created a variant of this dataset that disrupts the configuration of each object while preserving local features. While human performance was impacted by this manipulation, DCNN performance was not, indicating insensitivity to object configuration. Modifications to training and architecture to make networks more brain-like did not lead to configural processing, and none of the networks were able to accurately predict trial-by-trial human object judgements. We speculate that to match human configural sensitivity, networks must be trained to solve a broader range of object tasks beyond category recognition.}
        }

Ceiling

0.94.

Note that scores are relative to this ceiling.

Data: Baker2022fragmented

Metric: accuracy_delta