Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Baker2022fragmented-accuracy_delta")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.987
2
.987
3
.987
4
.986
5
.986
6
.986
7
.985
8
.984
9
.984
10
.984
11
.983
12
.982
13
.982
14
.982
15
.981
16
.978
17
.975
18
.970
19
.965
20
.964
21
.960
22
.960
23
.960
24
.959
25
.957
26
.954
27
.946
28
.945
29
.944
30
.944
31
.937
32
.935
33
.926
34
.925
35
.917
36
.909
37
.903
38
.901
39
.901
40
.901
41
.889
42
.882
43
.881
44
.868
45
.860
46
.858
47
.858
48
.838
49
.836
50
.836
51
.834
52
.832
53
.832
54
.822
55
.811
56
.806
57
.803
58
.802
59
.799
60
.796
61
.793
62
.791
63
.788
64
.787
65
.785
66
.774
67
.760
68
.758
69
.756
70
.755
71
.751
72
.740
73
.739
74
.735
75
.734
76
.734
77
.730
78
.723
79
.721
80
.720
81
.709
82
.700
83
.698
84
.691
85
.684
86
.671
87
.670
88
.663
89
.661
90
.659
91
.656
92
.649
93
.646
94
.626
95
.617
96
.603
97
.602
98
.597
99
.592
100
.590
101
.583
102
.582
103
.575
104
.574
105
.566
106
.558
107
.558
108
.551
109
.550
110
.543
111
.541
112
.538
113
.532
114
.528
115
.524
116
.523
117
.515
118
.507
119
.499
120
.494
121
.488
122
.478
123
.473
124
.470
125
.446
126
.445
127
.438
128
.433
129
.424
130
.421
131
.417
132
.412
133
.412
134
.412
135
.411
136
.400
137
.392
138
.392
139
.388
140
.365
141
.358
142
.350
143
.336
144
.336
145
.333
146
.332
147
.323
148
.315
149
.308
150
.308
151
.304
152
.289
153
.287
154
.282
155
.280
156
.274
157
.272
158
.268
159
.264
160
.251
161
.236
162
.224
163
.221
164
.217
165
.216
166
.204
167
.195
168
.195
169
.187
170
.186
171
.182
172
.178
173
.167
174
.166
175
.161
176
.154
177
.149
178
.140
179
.124
180
.116
181
.115
182
.111
183
.096
184
.096
185
.053
186
.039
187
.038
188
.032
189
.030
190
.029
191
.021
192
.015
193
.014
194
.011
195
.011
196
.003
197
.000
198
.000
199
.000
200
.000
201
.000
202
.000
203
.000
204
.000
205
.000
206
.000
207
.000
208
.000
209
.000
210
.000
211
.000
212
.000
213
.000
214
.000
215
.000
216
.000
217
.000
218
.000
219
.000
220
.000
221
.000
222
.000
223
.000
224
.000
225
.000
226
.000
227
.000
228
.000
229
.000
230
.000
231
.000
232
.000
233
.000
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336

Benchmark bibtex

@article{BAKER2022104913,
                title = {Deep learning models fail to capture the configural nature of human shape perception},
                journal = {iScience},
                volume = {25},
                number = {9},
                pages = {104913},
                year = {2022},
                issn = {2589-0042},
                doi = {https://doi.org/10.1016/j.isci.2022.104913},
                url = {https://www.sciencedirect.com/science/article/pii/S2589004222011853},
                author = {Nicholas Baker and James H. Elder},
                keywords = {Biological sciences, Neuroscience, Sensory neuroscience},
                abstract = {Summary
                A hallmark of human object perception is sensitivity to the holistic configuration of the local shape features of an object. Deep convolutional neural networks (DCNNs) are currently the dominant models for object recognition processing in the visual cortex, but do they capture this configural sensitivity? To answer this question, we employed a dataset of animal silhouettes and created a variant of this dataset that disrupts the configuration of each object while preserving local features. While human performance was impacted by this manipulation, DCNN performance was not, indicating insensitivity to object configuration. Modifications to training and architecture to make networks more brain-like did not lead to configural processing, and none of the networks were able to accurately predict trial-by-trial human object judgements. We speculate that to match human configural sensitivity, networks must be trained to solve a broader range of object tasks beyond category recognition.}
        }

Ceiling

0.94.

Note that scores are relative to this ceiling.

Data: Baker2022fragmented

Metric: accuracy_delta