2022 Brain-Score Competition

Our ability to recognize objects in the world relies on visual processing along the ventral visual stream, a set of hierarchically-organized cortical areas. Recently, neuroscientists and machine-learning researchers have produced computational models that have achieved moderate success at explaining primate object recognition behavior and the neural representations that support it. The first edition of the Brain-Score Competition proposes to evaluate how well these models can predict primate object recognition in 33 neuronal and behavioral benchmarks.

The Brain-Score Competition 2022 will be open to the scientific community and aims to accelerate the development of better models of primate vision. It will do so by providing an infrastructure to evaluate candidate models in a standardised and unified manner along a multitude of neuronal and behavioral benchmarks. In addition, it will incentivize model submissions by providing visibility to participants - via participation in a Cosyne workshop - and monetary prizes (total pool of $6,000) to the best performing models.

Submissions are open until February 15, 2022. For regular updates related to the competition, please follow Brain-Score on twitter and join our Slack channel. Good Luck!

Base v. Brain Models
Figure from Schrimpf et al. 2020

Overview

Participants should submit their models through the Brain-Score platform. Brain-Score currently accepts any image-computable model that can predict neural responses in ventral visual stream areas and/or object recognition behavior. While the overarching goal of the competition is to evaluate models that engage with the whole ventral visual stream, we also accept models that only attempt to model some of its components (see Competition Tracks below). Submitted models can be standard artificial neural networks (ANNs) for computer vision, which we call base models, or actual brain models with their internal components committed to specific visual areas, and their field-of-view mapped to a spatial visual extent (in visual degrees). To facilitate model submission, we provide helper wrappers for neural networks implemented in Pytorch, Tensorflow or Keras. For more information about model details, please check our tutorial.

Submitted models will be evaluated in a total of 33 neuronal and behavioral benchmarks related to activity in macaque visual cortical areas V1, V2, V4, and IT, and human psychophysical performance in a set of object classification tasks. Each benchmark contains a set of measurements as well as a comparative metric that evaluates how well the model can explain the experimental data. In the first edition of the Brain-Score Competition, the metrics will consist of a standard neural predictivity metric using PLS regression, the similarity of single-neuron response property distributions, and the alignment of object classification performance at the level of individual images (see Benchmarks below).

Competition Tracks

Main Track: V1, V2, V4, IT, and Behavior

  • First Prize: $2,000
  • Second Prize: $1,250
  • Third Prize: $750

The main goal of the Brain-Score Competition 2022 is to advance the development of models of the primate ventral visual stream and object recognition behavior. We believe that modelling this system as a whole, holds better promise for success than efforts focused solely on modelling specific components. Thus, the main competition track will simultaneously evaluate models in all the neuronal benchmarks for all the ventral stream areas as well as in a behavioral benchmark. We will award the three models with the highest overall scores, which consists of the average of five components: V1, V2, V4, IT, and behavior.

V1 Track

  • First Prize: $1,000

There is a large and vibrant community in neuroscience whose main focus is to model neuronal responses in the primary visual cortex (V1). For this reason, we will award the best V1 model from all the submissions. The V1 score is the average of a standard neural predictivity benchmark and a composite properties benchmark (see Benchmarks below).

Behavior Track

  • First Prize: $1,000

Cognitive scientists and machine learning researchers may be interested in modeling human object recognition behavior independently of the neural mechanisms that support it. We will award the best model in predicting human behavioral performance in a battery of object classification task.

See the official rules for full details.

Benchmarks

sample dicarlo.SanghaviMurty2020.V4-pls_v1/21.png sample movshon.FreemanZiemba2013.V1-pls_v2/26.png sample movshon.FreemanZiemba2013.V2-pls_v2/9.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/20.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/8.png sample dicarlo.Rajalingham2018-i2n_v2/7.png sample dicarlo.MajajHong2015.IT-pls_v3/19.png sample dicarlo.MajajHong2015.IT-pls_v3/5.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/25.png sample dicarlo.Sanghavi2020.IT-pls_v1/28.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/18.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/16.png sample movshon.FreemanZiemba2013.V2-pls_v2/13.png sample dicarlo.SanghaviMurty2020.V4-pls_v1/27.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/3.png sample movshon.FreemanZiemba2013.V1-pls_v2/27.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/14.png sample movshon.FreemanZiemba2013.V1-pls_v2/19.png sample movshon.FreemanZiemba2013.V2-pls_v2/22.png sample dicarlo.Rajalingham2018-i2n_v2/25.png sample dicarlo.MajajHong2015.IT-pls_v3/26.png sample dicarlo.Rajalingham2018-i2n_v2/28.png sample movshon.FreemanZiemba2013.V2-pls_v2/3.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/27.png sample dicarlo.MajajHong2015.IT-pls_v3/11.png sample dicarlo.MajajHong2015.V4-pls_v3/13.png sample dicarlo.Sanghavi2020.IT-pls_v1/6.png sample dicarlo.Rajalingham2018-i2n_v2/21.png sample movshon.FreemanZiemba2013.V1-pls_v2/8.png sample dicarlo.MajajHong2015.V4-pls_v3/3.png sample dicarlo.MajajHong2015.V4-pls_v3/6.png sample movshon.FreemanZiemba2013.V1-pls_v2/3.png sample dicarlo.MajajHong2015.IT-pls_v3/10.png sample dicarlo.Sanghavi2020.IT-pls_v1/0.png sample dicarlo.MajajHong2015.V4-pls_v3/21.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/3.png sample dicarlo.MajajHong2015.IT-pls_v3/12.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/1.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/26.png sample movshon.FreemanZiemba2013.V1-pls_v2/25.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/16.png sample movshon.FreemanZiemba2013.V2-pls_v2/7.png sample dicarlo.Rajalingham2018-i2n_v2/15.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/28.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/24.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/24.png sample dicarlo.MajajHong2015.IT-pls_v3/25.png sample dicarlo.SanghaviMurty2020.V4-pls_v1/7.png sample dicarlo.Rajalingham2018-i2n_v2/23.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/2.png sample dicarlo.Rajalingham2018-i2n_v2/18.png sample movshon.FreemanZiemba2013.V2-pls_v2/0.png sample movshon.FreemanZiemba2013.V2-pls_v2/26.png sample dicarlo.MajajHong2015.V4-pls_v3/18.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/27.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/14.png sample dicarlo.SanghaviMurty2020.V4-pls_v1/22.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/19.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/6.png sample dicarlo.SanghaviMurty2020.V4-pls_v1/13.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/5.png sample dicarlo.Sanghavi2020.IT-pls_v1/11.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/1.png sample dicarlo.SanghaviMurty2020.V4-pls_v1/5.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/9.png sample dicarlo.MajajHong2015.V4-pls_v3/9.png sample movshon.FreemanZiemba2013.V2-pls_v2/1.png sample movshon.FreemanZiemba2013.V1-pls_v2/2.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/25.png sample movshon.FreemanZiemba2013.V2-pls_v2/25.png sample dicarlo.Sanghavi2020.V4-pls_v1/27.png sample movshon.FreemanZiemba2013.V1-pls_v2/11.png sample dicarlo.Sanghavi2020.V4-pls_v1/6.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/18.png sample dicarlo.Sanghavi2020.V4-pls_v1/13.png sample dicarlo.Sanghavi2020.V4-pls_v1/1.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/2.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/29.png sample dicarlo.Sanghavi2020.IT-pls_v1/14.png sample dicarlo.Rajalingham2018-i2n_v2/2.png sample dicarlo.SanghaviJozwik2020.IT-pls_v1/4.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/15.png sample dicarlo.MajajHong2015.IT-pls_v3/1.png sample dicarlo.MajajHong2015.IT-pls_v3/27.png sample dicarlo.MajajHong2015.IT-pls_v3/22.png sample dicarlo.MajajHong2015.V4-pls_v3/2.png sample movshon.FreemanZiemba2013.V1-pls_v2/13.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/6.png sample movshon.FreemanZiemba2013.V1-pls_v2/0.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/22.png sample movshon.FreemanZiemba2013.V1-pls_v2/15.png sample dicarlo.MajajHong2015.V4-pls_v3/11.png sample dicarlo.MajajHong2015.V4-pls_v3/15.png sample dicarlo.Rajalingham2018-i2n_v2/1.png sample movshon.FreemanZiemba2013.V2-pls_v2/20.png sample dicarlo.MajajHong2015.V4-pls_v3/0.png sample dicarlo.SanghaviMurty2020.V4-pls_v1/28.png sample dicarlo.Rajalingham2018-i2n_v2/27.png sample dicarlo.Sanghavi2020.IT-pls_v1/26.png sample movshon.FreemanZiemba2013.V2-pls_v2/24.png sample dicarlo.Sanghavi2020.IT-pls_v1/13.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/23.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/29.png sample dicarlo.MajajHong2015.IT-pls_v3/17.png sample dicarlo.Sanghavi2020.V4-pls_v1/29.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/15.png sample dicarlo.MajajHong2015.IT-pls_v3/13.png sample dicarlo.SanghaviMurty2020.V4-pls_v1/1.png sample dicarlo.Sanghavi2020.IT-pls_v1/21.png sample movshon.FreemanZiemba2013.V1-pls_v2/5.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/3.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/9.png sample movshon.FreemanZiemba2013.V2-pls_v2/14.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/21.png sample dicarlo.Sanghavi2020.IT-pls_v1/23.png sample dicarlo.MajajHong2015.IT-pls_v3/8.png sample dicarlo.Sanghavi2020.V4-pls_v1/8.png sample dicarlo.Rajalingham2018-i2n_v2/3.png sample dicarlo.Sanghavi2020.IT-pls_v1/20.png sample dicarlo.Sanghavi2020.IT-pls_v1/16.png sample movshon.FreemanZiemba2013.V1-pls_v2/24.png sample movshon.FreemanZiemba2013.V1-pls_v2/1.png sample movshon.FreemanZiemba2013.V1-pls_v2/18.png sample dicarlo.MajajHong2015.V4-pls_v3/19.png sample dicarlo.Sanghavi2020.V4-pls_v1/11.png sample dicarlo.Sanghavi2020.V4-pls_v1/12.png sample dicarlo.Sanghavi2020.IT-pls_v1/4.png sample dicarlo.MajajHong2015.V4-pls_v3/25.png sample dicarlo.Sanghavi2020.IT-pls_v1/19.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/0.png sample dicarlo.Rajalingham2018-i2n_v2/16.png sample dicarlo.Sanghavi2020.V4-pls_v1/23.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/4.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/2.png sample dicarlo.SanghaviMurty2020.V4-pls_v1/2.png sample dicarlo.MajajHong2015.IT-pls_v3/20.png sample dicarlo.Sanghavi2020.V4-pls_v1/16.png sample dicarlo.MajajHong2015.V4-pls_v3/16.png sample dicarlo.Rajalingham2018-i2n_v2/22.png sample dicarlo.Sanghavi2020.IT-pls_v1/3.png sample movshon.FreemanZiemba2013.V1-pls_v2/4.png sample dicarlo.Sanghavi2020.IT-pls_v1/8.png sample dicarlo.MajajHong2015.IT-pls_v3/3.png sample dicarlo.Sanghavi2020.IT-pls_v1/17.png sample dicarlo.SanghaviJozwik2020.V4-pls_v1/5.png sample dicarlo.MajajHong2015.V4-pls_v3/27.png sample dicarlo.Rajalingham2018-i2n_v2/26.png sample dicarlo.Rajalingham2018-i2n_v2/5.png sample movshon.FreemanZiemba2013.V1-pls_v2/7.png sample dicarlo.SanghaviMurty2020.IT-pls_v1/16.png

The Brain-Score Competition 2022 will evaluate submitted models in 33 neuronal and behavioral benchmarks related to primate object recognition behavior. These benchmarks contain the following metrics:
  • Neural predictivity (image-level consistency with PLS regression): Measures how well the responses to given images in a model area predict the responses of a neuronal population of the corresponding area in the macaque brain. First, the model responses are mapped to the neuronal recordings using a linear transformation (PLS regression with 25 components) on a training set of images. Then the model’s predictivity is determined for held-out images by computing the Pearson correlation coefficient between the model’s predictions and the neuronal responses. This procedure is performed using cross-validation in 10 splits. More details about this neural predictivity metric can be found in Schrimpf, Kubilius, et al. 2018 and Yamins, Hong, et al. 2014. There is one neural predictivity benchmark for each area of the ventral stream in the macaque brain. The V1 and V2 benchmarks were built using neuronal datasets first described in Freeman, Ziemba, et al. 2013, and the V4 and IT benchmarks were built from recordings reported in Majaj, Hong, et al. 2015, Sanghavi and DiCarlo 2020, Sanghavi, Jozwik and DiCarlo 2020, and Sanghavi, Murty and DiCarlo 2020.
  • Single-neuron property distribution similarity: Measures whether single neurons in a model area are functionally similar to single-neurons in the corresponding monkey brain area. This is done by comparing the distribution of single-neuron response properties between the model area and the brain area using a similarity score (using the KS distance). For more details about this metric, please see Marques et al. 2021. There are 22 single-neuron property distribution similarity benchmarks corresponding to different response properties in the monkey V1. These single-neuron property distributions relate to several aspects of V1 responses and are pooled in seven groups: orientation tuning, spatial frequency tuning, receptive-field size, surround modulation, texture modulation, response selectivity, and response magnitude.
  • Behavioral consistency (normalized image-level): Measures the behavioral similarity between the model and humans in core object recognition tasks. This metric does not measure the overall accuracy of the model but whether it can predict the patterns of successes and failures of humans in a set of object recognition tasks. Model’s and humans’ behavioral accuracies are first transformed to a d’ statistic and then compared using the Pearson correlation coefficient. For more details about this behavioral metric, please see Rajalingham, Issa, et al. 2018 and Schrimpf, Kubilius, et al. 2018.

Competition Leaderboard

Rank
 

Model
submitted by

average

V1 23 benchmarks

V2 1 benchmark

V4 4 benchmarks

IT 4 benchmarks

behavior 1 benchmark

1
Anonymous Submitter
.493 .478 .249 .706 .749 .668 .958 .622 .516 .909 .793 .780 .621 .744 .587 .532 .710 .724 .732 .826 .559 .703 .793 .613 .873 .873 .472 .517 .427 .814 .704 .836 .902 .327 .327 .510 .611 .664 .504 .261 .494 .541 .557 .518 .359 .655 .655
2
Anonymous Submitter
.490 .490 .262 .718 .753 .863 .769 .617 .555 .844 .934 .688 .680 .860 .523 .657 .700 .751 .791 .786 .471 .671 .742 .600 .640 .640 .681 .755 .607 .902 .832 .936 .939 .314 .314 .509 .608 .670 .504 .253 .492 .540 .559 .514 .357 .642 .642
3
Anonymous Submitter
.489 .499 .263 .735 .733 .783 .745 .607 .579 .927 .816 .673 .739 .895 .566 .755 .762 .826 .884 .858 .479 .750 .792 .707 .660 .660 .631 .702 .560 .871 .851 .856 .907 .306 .306 .505 .606 .663 .502 .251 .487 .537 .556 .509 .346 .646 .646
4
Anonymous Submitter
.484 .495 .247 .744 .857 .939 .879 .774 .766 .805 .959 .873 .783 .886 .696 .767 .720 .821 .797 .826 .435 .791 .852 .730 .533 .533 .623 .720 .526 .902 .941 .865 .898 .307 .307 .502 .598 .652 .500 .257 .476 .532 .549 .506 .316 .642 .642
5
Anonymous Submitter
.482 .482 .242 .722 .892 .915 .956 .817 .805 .933 .928 .889 .758 .828 .664 .783 .694 .841 .864 .798 .273 .754 .855 .652 .610 .610 .565 .671 .459 .779 .694 .722 .922 .291 .291 .499 .605 .650 .487 .252 .476 .556 .535 .448 .364 .664 .664
6
Anonymous Submitter
.482 .532 .341 .723 .876 .949 1.0 .800 .734 .772 .988 .891 .910 .915 .962 .852 .790 .873 .901 .660 .726 .329 .461 .197 .385 .385 .885 .802 .968 .883 .884 .906 .859 .306 .306 .482 .583 .647 .492 .208 .504 .561 .562 .539 .353 .586 .586
7
Anonymous Submitter
.480 .479 .239 .719 .894 .917 .953 .787 .770 .907 .982 .941 .711 .711 .631 .792 .713 .798 .799 .793 .462 .780 .879 .680 .502 .502 .506 .631 .381 .927 .870 .933 .978 .294 .294 .501 .602 .656 .495 .252 .481 .529 .545 .506 .343 .643 .643
8
Anonymous Submitter
.476 .481 .227 .734 .887 .866 .970 .896 .844 .910 .887 .837 .755 .826 .663 .775 .725 .783 .747 .921 .448 .718 .844 .593 .547 .547 .633 .744 .521 .876 .812 .863 .952 .296 .296 .500 .600 .654 .502 .246 .477 .528 .545 .493 .343 .628 .628
9
Anonymous Submitter
.476 .499 .263 .735 .732 .784 .744 .607 .578 .926 .812 .672 .738 .896 .565 .753 .762 .827 .884 .857 .479 .749 .793 .706 .662 .662 .632 .702 .561 .872 .852 .856 .908 .306 .306 .505 .606 .663 .502 .251 .487 .537 .556 .509 .346 .580 .580
10
Anonymous Submitter
.473 .484 .254 .715 .810 .852 .860 .676 .640 .899 .939 .805 .707 .690 .599 .831 .757 .805 .810 .955 .458 .673 .744 .603 .526 .526 .586 .718 .453 .943 .909 .967 .954 .311 .311 .487 .589 .647 .487 .227 .491 .534 .551 .523 .357 .590 .590
11
Anonymous Submitter
.472 .474 .240 .709 .805 .831 .854 .692 .630 .890 .966 .775 .681 .764 .561 .719 .736 .809 .871 .804 .458 .749 .816 .682 .510 .510 .553 .671 .434 .926 .883 .946 .950 .294 .294 .500 .600 .657 .500 .246 .449 .532 .525 .435 .304 .644 .644
12
Anonymous Submitter
.471 .490 .262 .718 .753 .864 .771 .616 .554 .844 .934 .685 .680 .857 .527 .656 .698 .744 .789 .790 .471 .669 .739 .599 .639 .639 .683 .758 .609 .901 .829 .936 .937 .314 .314 .509 .608 .670 .504 .253 .492 .540 .559 .514 .357 .551 .551
13
Anonymous Submitter
.469 .512 .266 .758 .846 .909 .852 .718 .737 .888 .966 .853 .843 .781 .885 .862 .705 .765 .704 .965 .386 .823 .903 .744 .735 .735 .532 .662 .403 .825 .757 .757 .961 .313 .313 .481 .583 .630 .482 .228 .487 .535 .554 .500 .357 .550 .550
14
Anonymous Submitter
.467 .509 .252 .766 .867 .884 .957 .836 .830 .931 .720 .914 .761 .931 .742 .610 .763 .866 .811 .940 .436 .749 .834 .664 .743 .743 .640 .753 .527 .840 .745 .833 .943 .303 .303 .479 .593 .625 .458 .240 .484 .532 .545 .491 .368 .562 .562
15
Anonymous Submitter
.467 .484 .255 .713 .822 .860 .874 .701 .682 .914 .912 .813 .728 .714 .623 .846 .763 .799 .791 .966 .497 .668 .734 .602 .510 .510 .566 .703 .429 .935 .876 .965 .964 .299 .299 .485 .588 .640 .486 .224 .490 .530 .551 .523 .357 .576 .576
16
Anonymous Submitter
.464 .484 .246 .722 .787 .923 .838 .633 .627 .788 .928 .776 .714 .753 .651 .740 .699 .744 .731 .838 .484 .790 .809 .771 .491 .491 .701 .739 .662 .869 .814 .876 .918 .313 .313 .507 .610 .650 .493 .276 .485 .534 .550 .511 .345 .532 .532
17
Anonymous Submitter
.463 .477 .279 .675 .874 .970 .986 .797 .906 .812 .961 .684 .833 .769 .854 .877 .559 .528 .725 .220 .764 .194 .106 .282 .746 .746 .831 .887 .776 .688 .566 .526 .972 .336 .336 .470 .574 .625 .455 .228 .477 .537 .548 .484 .341 .555 .555
18
Anonymous Submitter
.463 .485 .265 .706 .852 .850 .879 .753 .726 .912 .943 .898 .724 .732 .660 .781 .639 .676 .817 .641 .424 .790 .882 .698 .497 .497 .515 .652 .377 .922 .901 .896 .970 .295 .295 .490 .583 .643 .497 .237 .455 .517 .534 .468 .300 .591 .591
19
Anonymous Submitter
.459 .469 .219 .719 .826 .880 .986 .808 .731 .631 .798 .950 .669 .565 .909 .533 .695 .839 .690 .886 .363 .650 .781 .518 .792 .792 .578 .698 .458 .824 .705 .803 .963 .309 .309 .471 .574 .621 .458 .233 .471 .529 .541 .467 .345 .573 .573
20
Anonymous Submitter
.456 .538 .355 .721 .890 .835 .947 .915 .872 .779 .985 .896 .901 .862 .985 .856 .679 .661 .688 .916 .449 .693 .794 .591 .536 .536 .468 .562 .375 .878 .786 .874 .974 .336 .336 .485 .620 .646 .451 .222 .459 .518 .514 .452 .354 .461 .461
21
Anonymous Submitter
.455 .424 .232 .617 .720 .644 .986 .714 .710 .434 .649 .902 .441 .791 .209 .322 .667 .746 .789 .766 .368 .614 .768 .459 .714 .714 .548 .650 .446 .616 .427 .584 .835 .318 .318 .488 .588 .641 .487 .236 .478 .530 .541 .487 .352 .569 .569
22
Anonymous Submitter
.455 .438 .234 .642 .656 .742 .672 .484 .421 .750 .848 .676 .598 .851 .605 .340 .706 .874 .855 .747 .349 .500 .474 .526 .662 .662 .572 .686 .459 .800 .738 .760 .901 .332 .332 .476 .574 .631 .463 .233 .483 .539 .551 .495 .348 .545 .545
23
Anonymous Submitter
.453 .443 .219 .667 .731 .644 .986 .644 .840 .436 .749 .818 .738 .769 .773 .672 .683 .792 .736 .851 .351 .612 .770 .453 .699 .699 .559 .635 .482 .647 .452 .653 .836 .307 .307 .486 .578 .639 .491 .235 .478 .523 .542 .509 .339 .550 .550
24
Anonymous Submitter
.452 .455 .236 .673 .844 .789 .899 .791 .804 .942 .827 .856 .602 .682 .393 .732 .659 .732 .648 .784 .472 .625 .591 .658 .705 .705 .503 .610 .397 .772 .750 .709 .858 .317 .317 .481 .574 .636 .495 .219 .479 .531 .548 .503 .332 .528 .528
25
Anonymous Submitter
.448 .487 .275 .699 .903 .953 .999 .837 .897 .884 .963 .789 .829 .680 .853 .953 .609 .595 .807 .189 .843 .239 .115 .362 .726 .726 .850 .899 .802 .734 .644 .592 .967 .294 .294 .442 .576 .608 .389 .195 .481 .536 .547 .488 .354 .536 .536
26
Anonymous Submitter
.427 .453 .227 .680 .859 .862 .877 .771 .753 .921 .951 .876 .789 .893 .743 .730 .559 .491 .528 .667 .549 .574 .701 .447 .550 .550 .529 .671 .387 .899 .796 .926 .976 .251 .251 .468 .576 .634 .503 .160 .407 .482 .477 .402 .269 .553 .553
27
Anonymous Submitter
.424 .508 .316 .700 .899 .876 .900 .865 .911 .868 .931 .939 .859 .933 .841