Researchers team of UC San Diego State University has developed a pair of “4-D goggles” that allows wearers to be physically “touched” a looming object on the movie screen
4-D goggles were developed based on a study by neuroscientists to map brain areas that combines the sight and touch of an appearing object and supports in the conception of the perceptual and neural mechanism of multisensory integration.
Researchers said, “The device can be synchronized with entertainment content, such as movies, music, games, and virtual reality, to deliver immersive multisensory effects near the face and enhance the sense of presence”.
Further described in an online paper published on February 6 in the journal Human Brain Mapping by Ruey-Song Huang and Ching-fu Chen, neuroscientists at UC San Diego’s Institute for Neural Computation, and Martin Sereno, the former chair of neuroimaging at University College London and a former professor at UC San Diego, now at San Diego State University.
“We perceive and interact with the world around us through multiple senses in daily life,” said Huang, the first author of the paper. “Though an approaching object may generate visual, auditory, and tactile signals in an observer, these must be picked apart from the rest of world, originally colorfully described by William James as a ‘blooming buzzing confusion.’ To detect and avoid impending threats, it is essential to integrate and analyze multisensory looming signals across space and time and to determine whether they originate from the same sources.”
While experimenting, subjects analyses the subjective coordination between a looming ball (simulated in virtual reality) and an air puff delivered to the same side of the face. When the arrival of ball movement and air puff were nearly concurrent (with a delay of 100 milliseconds), the air puff was recognized as completely in conflict with the looming ball. While with a delay near to 1000 millisecond, the two stimuli were recognized as one, like if an object passed neat the face, generating a little wind.
Using functional Magnetic Resonance Imaging, or fMRI in experiments, the scientists provided tactile-only, visual-only, tactile-visual out-of-sync, and tactile-visual in-sync stimuli to another side of the subject’s face in randomized events. Dozens of brain areas respond more strongly to lateralize multisensory stimuli than to lateralized unisensory stimuli, and the response was further enhanced when the multisensory stimuli are in perceptual sync, reported by the scientists in the paper.
The research was assisted by the National Institutes of Health (R01 MH081990), a Royal Society Wolfson Research Merit Award (UK), Wellcome Trust (UK), and a UC San Diego Frontiers of Innovation Scholars Program Project Fellowship.
