Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("ImageNet-C-blur-top1")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.685
2
.674
3
.650
4
.636
5
.626
6
.623
7
.612
8
.608
9
.596
10
.576
11
.560
12
.556
13
.548
14
.542
15
.538
16
.533
17
.528
18
.525
19
.525
20
.525
21
.516
22
.514
23
.514
24
.510
25
.507
26
.505
27
.503
28
.500
29
.494
30
.483
31
.475
32
.474
33
.471
34
.468
35
.463
36
.463
37
.456
38
.455
39
.448
40
.444
41
.442
42
.436
43
.434
44
.433
45
.431
46
.431
47
.426
48
.416
49
.416
50
.414
51
.413
52
.413
53
.412
54
.409
55
.406
56
.404
57
.404
58
.403
59
.399
60
.396
61
.395
62
.392
63
.391
64
.390
65
.388
66
.388
67
.388
68
.385
69
.384
70
.383
71
.383
72
.382
73
.381
74
.381
75
.379
76
.379
77
.379
78
.378
79
.377
80
.375
81
.375
82
.371
83
.370
84
.368
85
.366
86
.366
87
.366
88
.365
89
.364
90
.363
91
.362
92
.361
93
.361
94
.359
95
.357
96
.357
97
.357
98
.357
99
.356
100
.355
101
.355
102
.354
103
.352
104
.352
105
.351
106
.351
107
.351
108
.349
109
.349
110
.349
111
.348
112
.346
113
.345
114
.345
115
.343
116
.340
117
.339
118
.335
119
.331
120
.330
121
.329
122
.329
123
.326
124
.324
125
.324
126
.321
127
.321
128
.320
129
.312
130
.312
131
.312
132
.312
133
.307
134
.306
135
.306
136
.306
137
.306
138
.305
139
.301
140
.298
141
.297
142
.295
143
.294
144
.292
145
.289
146
.288
147
.287
148
.285
149
.284
150
.282
151
.280
152
.279
153
.276
154
.275
155
.274
156
.273
157
.271
158
.267
159
.266
160
.264
161
.263
162
.260
163
.260
164
.260
165
.259
166
.259
167
.259
168
.258
169
.258
170
.257
171
.256
172
.256
173
.255
174
.255
175
.252
176
.252
177
.249
178
.248
179
.248
180
.244
181
.240
182
.239
183
.239
184
.237
185
.237
186
.234
187
.232
188
.231
189
.231
190
.229
191
.229
192
.229
193
.225
194
.218
195
.218
196
.216
197
.214
198
.214
199
.213
200
.211
201
.210
202
.207
203
.206
204
.206
205
.204
206
.202
207
.201
208
.196
209
.195
210
.193
211
.192
212
.192
213
.192
214
.192
215
.191
216
.191
217
.191
218
.191
219
.187
220
.187
221
.187
222
.187
223
.187
224
.185
225
.183
226
.181
227
.179
228
.179
229
.179
230
.179
231
.179
232
.178
233
.175
234
.173
235
.171
236
.170
237
.168
238
.165
239
.163
240
.156
241
.155
242
.154
243
.154
244
.146
245
.144
246
.132
247
.132
248
.130
249
.127
250
.127
251
.120
252
.119
253
.108
254
.102
255
.020
256
.020
257
.006
258
.003
259
.003
260
.002
261
.002
262
.002
263
.001
264
.001
265
.001
266
.001
267
.001
268
.001
269
.001
270
.001
271
.001
272
.001
273
.001
274
.001
275
.001
276
.001
277
.001
278
.001
279
.001
280
.001
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482

Benchmark bibtex

@ARTICLE{Hendrycks2019-di,
   title         = "Benchmarking Neural Network Robustness to Common Corruptions
                    and Perturbations",
   author        = "Hendrycks, Dan and Dietterich, Thomas",
   abstract      = "In this paper we establish rigorous benchmarks for image
                    classifier robustness. Our first benchmark, ImageNet-C,
                    standardizes and expands the corruption robustness topic,
                    while showing which classifiers are preferable in
                    safety-critical applications. Then we propose a new dataset
                    called ImageNet-P which enables researchers to benchmark a
                    classifier's robustness to common perturbations. Unlike
                    recent robustness research, this benchmark evaluates
                    performance on common corruptions and perturbations not
                    worst-case adversarial perturbations. We find that there are
                    negligible changes in relative corruption robustness from
                    AlexNet classifiers to ResNet classifiers. Afterward we
                    discover ways to enhance corruption and perturbation
                    robustness. We even find that a bypassed adversarial defense
                    provides substantial common perturbation robustness.
                    Together our benchmarks may aid future work toward networks
                    that robustly generalize.",
   month         =  mar,
   year          =  2019,
   archivePrefix = "arXiv",
   primaryClass  = "cs.LG",
   eprint        = "1903.12261",
   url           = "https://arxiv.org/abs/1903.12261"
}

Ceiling

1.00.

Note that scores are relative to this ceiling.

Data: ImageNet-C-blur

Metric: top1