Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Kar2019-ost")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.310
2
.306
3
.305
4
.297
5
.290
6
.276
7
.273
8
.272
9
.270
10
.267
11
.267
12
.260
13
.259
14
.258
15
.255
16
.255
17
.253
18
.252
19
.251
20
.247
21
.240
22
.233
23
.232
24
.218
25
.142
26
1.0
27
1.0
28
1.0
29
1.0
30
1.0
31
1.0
32
1.0
33
1.0
34
1.0
35
1.0
36
1.0
37
1.0
38
1.0
39
1.0
40
1.0
41
1.0
42
1.0
43
1.0
44
1.0
45
1.0
46
1.0
47
1.0
48
1.0
49
1.0
50
1.0
51
1.0
52
1.0
53
1.0
54
1.0
55
1.0
56
1.0
57
1.0
58
1.0
59
1.0
60
1.0
61
1.0
62
1.0
63
1.0
64
1.0
65
1.0
66
1.0
67
1.0
68
1.0
69
1.0
70
1.0
71
1.0
72
1.0
73
1.0
74
1.0
75
1.0
76
1.0
77
1.0
78
1.0
79
1.0
80
1.0
81
1.0
82
1.0
83
1.0
84
1.0
85
1.0
86
1.0
87
1.0
88
1.0
89
1.0
90
1.0
91
1.0
92
1.0
93
1.0
94
1.0
95
1.0
96
1.0
97
1.0
98
1.0
99
1.0
100
1.0
101
1.0
102
1.0
103
104
1.0
105
1.0
106
1.0
107
1.0
108
1.0
109
1.0
110
1.0
111
1.0
112
1.0
113
114
115
116
117
118
119
1.0
120
1.0
121
122
1.0
123
124
1.0
125
126
1.0
127
1.0
128
1.0
129
1.0
130
1.0
131
1.0
132
1.0
133
1.0
134
1.0
135
1.0
136
1.0
137
1.0
138
1.0
139
1.0
140
1.0
141
1.0
142
1.0
143
1.0
144
1.0
145
1.0
146
1.0
147
1.0
148
1.0
149
1.0
150
1.0
151
1.0
152
1.0
153
1.0
154
1.0
155
1.0
156
1.0
157
1.0
158
1.0
159
1.0
160
1.0
161
1.0
162
1.0
163
1.0
164
1.0
165
1.0
166
1.0
167
1.0
168
1.0
169
1.0
170
1.0
171
1.0
172
1.0
173
1.0
174
1.0
175
1.0
176
1.0
177
1.0
178
1.0
179
1.0
180
1.0
181
1.0
182
1.0
183
1.0
184
1.0
185
1.0
186
1.0
187
1.0
188
1.0
189
1.0
190
1.0
191
1.0
192
1.0
193
1.0
194
1.0
195
1.0
196
1.0
197
1.0
198
1.0
199
1.0
200
1.0
201
1.0
202
1.0
203
1.0
204
1.0
205
1.0
206
1.0
207
1.0
208
1.0
209
1.0
210
1.0
211
1.0
212
1.0
213
1.0
214
1.0
215
1.0
216
1.0
217
1.0
218
1.0
219
1.0
220
1.0
221
1.0
222
1.0
223
1.0
224
1.0
225
1.0
226
1.0
227
1.0
228
1.0
229
1.0
230
1.0
231
1.0
232
1.0
233
1.0
234
1.0
235
236
1.0
237
1.0
238
1.0
239
1.0
240
1.0
241
1.0
242
1.0
243
1.0
244
1.0
245
1.0
246
1.0
247
1.0
248
1.0
249
1.0
250
1.0
251
1.0
252
1.0
253
1.0
254
1.0
255
1.0
256
1.0
257
1.0
258
1.0
259
1.0
260
1.0
261
1.0
262
1.0
263
1.0
264
1.0
265
1.0
266
1.0
267
1.0
268
1.0
269
1.0
270
1.0
271
1.0
272
1.0
273
274
1.0
275
1.0
276
1.0
277
1.0
278
1.0
279
1.0
280
1.0
281
1.0
282
1.0
283
1.0
284
1.0
285
1.0
286
1.0
287
1.0
288
1.0
289
1.0
290
1.0
291
1.0
292
1.0
293
1.0
294
1.0
295
1.0
296
1.0
297
1.0
298
1.0
299
300
1.0
301
1.0
302
1.0
303
1.0
304
1.0
305
1.0
306
1.0
307
1.0
308
1.0
309
1.0
310
1.0
311
1.0
312
1.0
313
1.0
314
1.0
315
1.0
316
1.0
317
1.0
318
1.0
319
1.0
320
1.0
321
1.0
322
1.0
323
1.0
324
1.0
325
1.0
326
1.0
327
1.0
328
1.0
329
1.0
330
1.0
331
1.0
332
1.0
333
1.0
334
1.0
335
1.0
336
337
338
339
340
341
1.0
342
1.0
343
1.0
344
1.0
345
1.0
346
1.0
347
1.0
348
1.0
349
1.0
350
1.0
351
352
353
1.0
354
355
1.0
356
1.0
357
1.0
358
1.0
359
1.0
360
1.0
361
1.0
362
1.0
363
1.0
364
1.0
365
1.0
366
1.0
367
1.0
368
1.0
369
1.0
370
1.0
371
1.0
372
1.0
373
1.0
374
1.0
375
1.0
376
1.0
377
1.0
378
1.0
379
1.0
380
1.0
381
1.0
382
1.0
383
1.0
384
1.0
385
1.0
386
1.0
387
1.0
388
1.0
389
1.0
390
1.0
391
1.0
392
1.0
393
1.0
394
1.0
395
1.0
396
1.0
397
1.0
398
1.0
399
1.0
400
401
1.0
402
1.0
403
1.0
404
1.0
405
1.0
406
1.0
407
1.0
408
1.0
409
1.0
410
1.0
411
1.0
412
1.0
413
1.0
414
1.0
415
1.0
416
1.0
417
418
1.0
419
1.0
420
1.0
421
1.0
422
423
1.0
424
1.0
425
1.0
426
1.0
427
1.0
428
1.0
429
1.0
430
1.0
431
1.0
432
1.0
433
1.0
434
1.0
435
1.0
436
1.0
437
438
1.0
439
1.0
440
1.0
441
1.0
442
1.0
443
1.0
444
445
446
1.0
447
1.0
448
1.0
449
1.0
450
451
452
453
1.0
454
1.0
455
1.0
456
1.0
457
1.0
458
1.0
459
1.0
460
1.0
461
1.0
462
1.0
463
1.0
464
1.0
465
1.0
466
1.0
467
1.0
468
1.0
469
1.0
470
1.0
471
1.0
472
1.0
473
1.0
474
1.0

Benchmark bibtex

@Article{Kar2019,
                                                    author={Kar, Kohitij
                                                    and Kubilius, Jonas
                                                    and Schmidt, Kailyn
                                                    and Issa, Elias B.
                                                    and DiCarlo, James J.},
                                                    title={Evidence that recurrent circuits are critical to the ventral stream's execution of core object recognition behavior},
                                                    journal={Nature Neuroscience},
                                                    year={2019},
                                                    month={Jun},
                                                    day={01},
                                                    volume={22},
                                                    number={6},
                                                    pages={974-983},
                                                    abstract={Non-recurrent deep convolutional neural networks (CNNs) are currently the best at modeling core object recognition, a behavior that is supported by the densely recurrent primate ventral stream, culminating in the inferior temporal (IT) cortex. If recurrence is critical to this behavior, then primates should outperform feedforward-only deep CNNs for images that require additional recurrent processing beyond the feedforward IT response. Here we first used behavioral methods to discover hundreds of these `challenge' images. Second, using large-scale electrophysiology, we observed that behaviorally sufficient object identity solutions emerged {	extasciitilde}30{	hinspace}ms later in the IT cortex for challenge images compared with primate performance-matched `control' images. Third, these behaviorally critical late-phase IT response patterns were poorly predicted by feedforward deep CNN activations. Notably, very-deep CNNs and shallower recurrent CNNs better predicted these late IT responses, suggesting that there is a functional equivalence between additional nonlinear transformations and recurrence. Beyond arguing that recurrent circuits are critical for rapid object identification, our results provide strong constraints for future recurrent model development.},
                                                    issn={1546-1726},
                                                    doi={10.1038/s41593-019-0392-5},
                                                    url={https://doi.org/10.1038/s41593-019-0392-5}
                                                    }

Ceiling

0.79.

Note that scores are relative to this ceiling.

Data: Kar2019

1318 stimuli recordings from 424 sites in IT

Metric: ost