3.08.2008
Inner Vision: Brain Reading Algorithm Can Predict What People are Seeing
>
-from National Geographic and Nature
A new computer program can match brain activity with visual images and even predict what people are seeing, a study has shown.
The work raises the possibility that one day computers could "read" a person's brain to digitally re-create memories, dreams, or imaginings.
Previous attempts to decode vision in this way could only extract simple information about images, such as their physical orientation, and could not identify images that participants were seeing for the first time.
"Our technique overcomes this limitation, and we show that we can perform identification for novel images," said study team member Kendrick Kay of the University of California, Berkeley.
The new computer model is described in the journal Nature.
The researchers used functional magnetic resonance imaging (fMRI) to measure activity in the visual cortices of participants' brains as they looked at photographs of animals, food, people, and other common objects.
The fMRI technique is a relatively new way to measure changes in the brain's blood oxygen levels, which have strong links to neural activity.
The collected data were used to "teach" a computer program to associate certain blood flow patterns with particular kinds of images.
Participants were then asked to look at a second set of images they had never encountered before.
The model was programmed to take what it had learned from the previous pairings and figure out what was being shown in the new set of images.
For a collection of 120 images, the model correctly identified what a person was looking at 90 percent of the time. When the set was enlarged to a thousand images, accuracy was about 80 percent.
Brain Readers
The researchers say their work opens the door for brain-reading devices—like those envisioned by Philip K. Dick and other science-fiction writers—that display a person's inner visual experiences on a screen.
Before such a device can be built, however, researchers must first answer important questions about dreams, memories, and imagination.
(Related: "First Ever Brain 'Atlas' Completed [September 26, 2006].)
"Perhaps the contents of our imaginations are not represented in the same way as the contents of our actual real perceptions," Kay said.
"In this case, we will have to investigate how imagination is represented and construct appropriate computational models."
Technology will have to improve as well.
Many critics of fMRI point out that the technique does not measure brain activity directly. As a result, it lacks the resolution of data recorded directly from brain cells.
Small Step
Frank Tong, a neuroscientist at Vanderbilt University in Tennessee, said he was surprised that the team's method worked as well as it did for this very reason.
"Most people think of fMRI as a pretty crude method, but [the data collected] contained a surprising amount of information, enough to predict, well above [the level of mere] chance, which of several hundred or thousand pictures a person was looking at," said Tong, who was not involved in the study.
The work also builds on other groundbreaking studies, including research reported last year in the journal Current Biology, in which researchers were able to decode the simple intentions of participants about 70 percent of the time based on fMRI readings.
Robert Dougherty, a neuroscientist at Stanford University, said the construction of a brain-reading device might be possible, but he cautions that the team's new model is only a small step toward that goal.
"Their model is not invertible—it cannot generate a unique image from the measured brain activity," Dougherty added.
"However, combined with strong assumptions about natural image statistics, a more sophisticated model could produce such images that would be a prediction of a subject's visual imagery."
A brain-reading device would be valuable for probing phenomena that are difficult to study using conventional means, such the differences in perception among people, the researchers said.
But the team notes that such a device could be used for more sinister purposes as well.
The privacy and ethical concerns associated with a brain-reading device would parallel those surrounding human genome sequencing, the researchers said.
In both cases, care will need to be taken so that the rights of individuals are not violated.
"The authors believe strongly that no one should be subjected to any form of brain-reading process involuntarily, covertly, or without complete informed consent," the team wrote in a statement.
A new computer program can match brain activity with visual images and even predict what people are seeing, a study has shown.
The work raises the possibility that one day computers could "read" a person's brain to digitally re-create memories, dreams, or imaginings.
Previous attempts to decode vision in this way could only extract simple information about images, such as their physical orientation, and could not identify images that participants were seeing for the first time.
"Our technique overcomes this limitation, and we show that we can perform identification for novel images," said study team member Kendrick Kay of the University of California, Berkeley.
The new computer model is described in the journal Nature.
The researchers used functional magnetic resonance imaging (fMRI) to measure activity in the visual cortices of participants' brains as they looked at photographs of animals, food, people, and other common objects.
The fMRI technique is a relatively new way to measure changes in the brain's blood oxygen levels, which have strong links to neural activity.
The collected data were used to "teach" a computer program to associate certain blood flow patterns with particular kinds of images.
Participants were then asked to look at a second set of images they had never encountered before.
The model was programmed to take what it had learned from the previous pairings and figure out what was being shown in the new set of images.
For a collection of 120 images, the model correctly identified what a person was looking at 90 percent of the time. When the set was enlarged to a thousand images, accuracy was about 80 percent.
Brain Readers
The researchers say their work opens the door for brain-reading devices—like those envisioned by Philip K. Dick and other science-fiction writers—that display a person's inner visual experiences on a screen.
Before such a device can be built, however, researchers must first answer important questions about dreams, memories, and imagination.
(Related: "First Ever Brain 'Atlas' Completed [September 26, 2006].)
"Perhaps the contents of our imaginations are not represented in the same way as the contents of our actual real perceptions," Kay said.
"In this case, we will have to investigate how imagination is represented and construct appropriate computational models."
Technology will have to improve as well.
Many critics of fMRI point out that the technique does not measure brain activity directly. As a result, it lacks the resolution of data recorded directly from brain cells.
Small Step
Frank Tong, a neuroscientist at Vanderbilt University in Tennessee, said he was surprised that the team's method worked as well as it did for this very reason.
"Most people think of fMRI as a pretty crude method, but [the data collected] contained a surprising amount of information, enough to predict, well above [the level of mere] chance, which of several hundred or thousand pictures a person was looking at," said Tong, who was not involved in the study.
The work also builds on other groundbreaking studies, including research reported last year in the journal Current Biology, in which researchers were able to decode the simple intentions of participants about 70 percent of the time based on fMRI readings.
Robert Dougherty, a neuroscientist at Stanford University, said the construction of a brain-reading device might be possible, but he cautions that the team's new model is only a small step toward that goal.
"Their model is not invertible—it cannot generate a unique image from the measured brain activity," Dougherty added.
"However, combined with strong assumptions about natural image statistics, a more sophisticated model could produce such images that would be a prediction of a subject's visual imagery."
A brain-reading device would be valuable for probing phenomena that are difficult to study using conventional means, such the differences in perception among people, the researchers said.
But the team notes that such a device could be used for more sinister purposes as well.
The privacy and ethical concerns associated with a brain-reading device would parallel those surrounding human genome sequencing, the researchers said.
In both cases, care will need to be taken so that the rights of individuals are not violated.
"The authors believe strongly that no one should be subjected to any form of brain-reading process involuntarily, covertly, or without complete informed consent," the team wrote in a statement.
Labels: cat, dick, kendrick, mind_games, minds, vision
