THEORETICAL FRAMEWORK: Vision and touch share a critical function—perception of 3D object shape. We use vision and touch both alternatively and simultaneously to recognize, understand, and interact physically with real world objects, based on detailed apprehension of their 3D shapes and mechanical functionality. METHODS, INNOVATION, SPECIES: We propose a novel exploration of visual/haptic circuit interactions underlying 3D object perception: (i) at the spatiotemporal resolution of individual neurons and action potentials, (ii) across the scope of local neural networks studied with linear array recording in monkey IT, and (iii) in the behavioral context of active grasp to discriminate unseen objects. AIM 1 TEST OF HYPOTHESIS: Haptic 3D object discrimination evokes distinct, shape-specific neural population response patterns in anterior monkey IT. This predicted result would provide strong evidence that the final stage in the visual object pathway also carries haptic-origin information sufficient to discriminate 3D shapes through touch, as suggested by human fMRI. AIM 2 TEST OF HYPOTHESIS: Haptic shape-dependent responses in monkey IT embody a compositional geometric code for 3D shape, analogous to the IT code for visual 3D shape. We have previously shown that IT neurons responding to 3D object stimuli encode volumetric shape fragments in a high-dimensional geometric space. These signals provide the essential elements for ensembles of IT neurons to represent objects as 3D spatial compositions. We predict analogous compositional coding for haptic 3D shape, showing that vision and touch converge on the same powerful coding solution. AIM 3 TEST OF HYPOTHESIS: Individual IT neurons carry identical 3D shape information about haptic and visual objects. This predicted result of congruent visual and haptic coding by the same neurons would demonstrate that anterior IT carries a unified, supramodal 3D object representation. ENABLING A FUTURE BCP R01, LONG-TERM GOAL: Success on any of these aims would justify a subsequent TargetedBCP R01 application to characterize HOW haptic and visual circuits interact to: (i) exchange information across circuits connecting multiple processing stages (visual areas V1, V2, V4, posterior IT, anterior IT; somatosensory areas SI, SII, superior parietal, insula). (ii) dynamically combine and refine 3D shape information across visual and haptic circuits as congruent, redundant, or conflicting information accumulates from the two modalities. (iii) interact in new object learning, since the two modalities almost certainly provide mutual supervision, especially during development, when vision must calibrate so many indirect cues for 3D shape information directly available to touch. The LONGTERM GOAL is to understand how visual and haptic cortical circuits refine, reconcile, and synthesize two very different sensory inputs to produce a unified cognitive appreciation of 3D shape.

This study investigates how visual and somatosensory circuits cooperate in perception, understanding, and learning about 3D shapes of real world objects. The results will elucidate for the first time the circuit level mechanisms behind our ability to use vision and touch independently or in cooperation to understand and interact with 3D objects. This will impact our development and use of visual and haptic prosthetics in ways that take maximal advantage of the circuit redundancies in the two systems.