PROJECT SUMMARY. To understand vision, we must be able to explain how it works in the natural world. Currently, while we can explain and predict how visual cortex neurons respond in limited laboratory conditions, most predictions fail when neurons are tested with randomly selected photographs of the natural world. One problem is that neurons in the monkey are usually tested using simple stimuli such as lines and dots, or using photographs from limited semantically defined categories (e.g. faces, places). In order to advance our knowledge of how neurons function with more complex visual inputs, we manipulated naturalistic images using neural network models called generative adversarial networks (GANs). GANS are trained to parameterize images similar to those in the natural world. We found that, when combined with evolutionary (search) algorithms, GANs synthesized images that highly activated inferotemporal cortex (IT) neurons, recorded using microelectrode arrays in awake behaving macaques. Neurons showed a range of response firing rates that exceed those elicited by previous approaches. The discovery of these highly activating images has energized a field-wide debate about how to best describe the tuning of neurons of the object-recognition system. Should neuronal activity be described using investigator-designed parametric frameworks (e.g., orientation, curvature), perceptual distances from highly activating images, or through data-fitted neural networks? In this proposed research, we will test these and other experimental designs to determine the best way to predict neuronal responses to natural images. We will pair approaches in a “tournament”-like meta-design, testing the same populations of neurons over hours and across days, using chronically implanted microelectrode arrays in awake, behaving non-human primates. We will also show which methods are best at predicting population response patterns comprising dozens of cortical visual sites. The project will include the development of workshops with other investigating teams in order to develop standard terminology and desiderata in explanatory theories of visual function. Although our own overarching theory is that the activity of a given cortical neuron represents the similarity from visual inputs to that neuron's most highly activating image (more precisely, to the visual attributes it contains), we will rely on neural networks as a unifying mechanism behind all approaches. The project will illustrate how to derive brain- wide organizational principles (based on ethology and concepts from natural selection) to explain visual recognition, and to constrain the space of neural networks that can best serve as models of the brain. The expected outcome is a framework for understanding how occipito-temporal neurons act in naturalistic image spaces, and how their representational capabilities inform recognition-based behaviors. Further, we will create code repositories (https://github.com/PonceLab) to encourage others to implement all approaches in their own research.

PUBLIC HEALTH RELEVANCE: The goal of this proposal is to explain and predict how visual system neurons function in naturalistic settings. The proposed research is relevant to public health because understanding the brain's strategies for solving problems of visual recognition will reveal fundamental aspects of neurological function, provide a target mechanism in sensory processing disorders (e.g. cortical visual impairments), and lead to efficient visual recognition systems needed in medical imaging and bioprosthetics for the visually impaired.