Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Baker2022fragmented-accuracy_delta")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.986
2
.984
3
.984
4
.983
5
.982
6
.982
7
.982
8
.981
9
.978
10
.970
11
.965
12
.960
13
.960
14
.960
15
.957
16
.946
17
.945
18
.944
19
.944
20
.935
21
.926
22
.925
23
.917
24
.903
25
.901
26
.901
27
.901
28
.889
29
.882
30
.868
31
.858
32
.858
33
.838
34
.836
35
.836
36
.834
37
.832
38
.822
39
.811
40
.806
41
.803
42
.802
43
.799
44
.796
45
.791
46
.788
47
.785
48
.760
49
.758
50
.756
51
.751
52
.740
53
.735
54
.734
55
.734
56
.730
57
.721
58
.720
59
.709
60
.698
61
.671
62
.670
63
.663
64
.656
65
.649
66
.646
67
.617
68
.603
69
.602
70
.592
71
.590
72
.583
73
.582
74
.566
75
.558
76
.558
77
.550
78
.543
79
.541
80
.538
81
.528
82
.524
83
.523
84
.515
85
.507
86
.499
87
.494
88
.478
89
.473
90
.470
91
.446
92
.438
93
.433
94
.424
95
.421
96
.417
97
.412
98
.412
99
.412
100
.411
101
.400
102
.392
103
.392
104
.388
105
.365
106
.350
107
.336
108
.336
109
.333
110
.308
111
.304
112
.289
113
.287
114
.282
115
.280
116
.274
117
.268
118
.264
119
.251
120
.236
121
.221
122
.217
123
.216
124
.204
125
.195
126
.195
127
.186
128
.178
129
.167
130
.161
131
.149
132
.115
133
.111
134
.096
135
.053
136
.038
137
.032
138
.030
139
.029
140
.021
141
.015
142
.014
143
.011
144
.011
145
.003
146
.000
147
.000
148
.000
149
.000
150
.000
151
.000
152
.000
153
.000
154
.000
155
.000
156
.000
157
.000
158
.000
159
.000
160
.000
161
.000
162
.000
163
.000
164
.000
165
.000
166
.000
167
.000
168
.000
169
.000
170
.000
171
.000
172
.000
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246

Benchmark bibtex

@article{BAKER2022104913,
                title = {Deep learning models fail to capture the configural nature of human shape perception},
                journal = {iScience},
                volume = {25},
                number = {9},
                pages = {104913},
                year = {2022},
                issn = {2589-0042},
                doi = {https://doi.org/10.1016/j.isci.2022.104913},
                url = {https://www.sciencedirect.com/science/article/pii/S2589004222011853},
                author = {Nicholas Baker and James H. Elder},
                keywords = {Biological sciences, Neuroscience, Sensory neuroscience},
                abstract = {Summary
                A hallmark of human object perception is sensitivity to the holistic configuration of the local shape features of an object. Deep convolutional neural networks (DCNNs) are currently the dominant models for object recognition processing in the visual cortex, but do they capture this configural sensitivity? To answer this question, we employed a dataset of animal silhouettes and created a variant of this dataset that disrupts the configuration of each object while preserving local features. While human performance was impacted by this manipulation, DCNN performance was not, indicating insensitivity to object configuration. Modifications to training and architecture to make networks more brain-like did not lead to configural processing, and none of the networks were able to accurately predict trial-by-trial human object judgements. We speculate that to match human configural sensitivity, networks must be trained to solve a broader range of object tasks beyond category recognition.}
        }

Ceiling

Not available

Data: Baker2022fragmented

Metric: accuracy_delta