Cognitive Labs

-from National Geographic and Nature




A new computer program can match brain activity with visual images and even predict what people are seeing, a study has shown.

The work raises the possibility that one day computers could "read" a person's brain to digitally re-create memories, dreams, or imaginings.

Previous attempts to decode vision in this way could only extract simple information about images, such as their physical orientation, and could not identify images that participants were seeing for the first time.

"Our technique overcomes this limitation, and we show that we can perform identification for novel images," said study team member Kendrick Kay of the University of California, Berkeley.

The new computer model is described in the journal Nature.

The researchers used functional magnetic resonance imaging (fMRI) to measure activity in the visual cortices of participants' brains as they looked at photographs of animals, food, people, and other common objects.

The fMRI technique is a relatively new way to measure changes in the brain's blood oxygen levels, which have strong links to neural activity.

The collected data were used to "teach" a computer program to associate certain blood flow patterns with particular kinds of images.

Participants were then asked to look at a second set of images they had never encountered before.

The model was programmed to take what it had learned from the previous pairings and figure out what was being shown in the new set of images.

For a collection of 120 images, the model correctly identified what a person was looking at 90 percent of the time. When the set was enlarged to a thousand images, accuracy was about 80 percent.

Brain Readers

The researchers say their work opens the door for brain-reading devices—like those envisioned by Philip K. Dick and other science-fiction writers—that display a person's inner visual experiences on a screen.

Before such a device can be built, however, researchers must first answer important questions about dreams, memories, and imagination.

(Related: "First Ever Brain 'Atlas' Completed [September 26, 2006].)

"Perhaps the contents of our imaginations are not represented in the same way as the contents of our actual real perceptions," Kay said.

"In this case, we will have to investigate how imagination is represented and construct appropriate computational models."

Technology will have to improve as well.

Many critics of fMRI point out that the technique does not measure brain activity directly. As a result, it lacks the resolution of data recorded directly from brain cells.

Small Step

Frank Tong, a neuroscientist at Vanderbilt University in Tennessee, said he was surprised that the team's method worked as well as it did for this very reason.

"Most people think of fMRI as a pretty crude method, but [the data collected] contained a surprising amount of information, enough to predict, well above [the level of mere] chance, which of several hundred or thousand pictures a person was looking at," said Tong, who was not involved in the study.

The work also builds on other groundbreaking studies, including research reported last year in the journal Current Biology, in which researchers were able to decode the simple intentions of participants about 70 percent of the time based on fMRI readings.

Robert Dougherty, a neuroscientist at Stanford University, said the construction of a brain-reading device might be possible, but he cautions that the team's new model is only a small step toward that goal.

"Their model is not invertible—it cannot generate a unique image from the measured brain activity," Dougherty added.

"However, combined with strong assumptions about natural image statistics, a more sophisticated model could produce such images that would be a prediction of a subject's visual imagery."

A brain-reading device would be valuable for probing phenomena that are difficult to study using conventional means, such the differences in perception among people, the researchers said.

But the team notes that such a device could be used for more sinister purposes as well.

The privacy and ethical concerns associated with a brain-reading device would parallel those surrounding human genome sequencing, the researchers said.

In both cases, care will need to be taken so that the rights of individuals are not violated.

"The authors believe strongly that no one should be subjected to any form of brain-reading process involuntarily, covertly, or without complete informed consent," the team wrote in a statement.

Labels: cat, dick, kendrick, mind_games, minds, vision

Memory and future thought go 'hand-in-hand'

Human memory, the ability to recall vivid mental images of past experiences, has been studied extensively for more than a hundred years. But until recently, there's been surprisingly little research into cognitive processes underlying another form of mental time travel -- the ability to clearly imagine or "see" oneself participating in a future event.

Now, researchers from Washington University in St. Louis have used advanced brain imaging techniques to show that remembering the past and envisioning the future may go hand-in-hand, with each process sparking strikingly similar patterns of activity within precisely the same broad network of brain regions.

"In our daily lives, we probably spend more time envisioning what we're going to do tomorrow or later on in the day than we do remembering, but not much is known about how we go about forming these mental images of the future," says Karl Szpunar, lead author of the study and a psychology doctoral student in Arts & Sciences at Washington University.

"Our findings provide compelling support for the idea that memory and future thought are highly interrelated and help explain why future thought may be impossible without memories."

Scheduled for advance online publication Jan. 1 in Proceedings of the National Academy of Sciences, the study sheds new light on how the human mind relies on the vivid recollection of past experiences to prepare itself for future challenges, suggesting that envisioning the future may be a critical prerequisite for many higher-level planning processes.

Other study co-authors are Jason M. Watson, a Washington University doctoral graduate now assistant professor of psychology at the University of Utah; and Kathleen McDermott, an associate professor of psychology in Arts & Sciences and of radiology in the School of Medicine at Washington University.

McDermott, principal investigator for the University's Memory and Cognition Lab, where the research is based, suggests that the findings are notable for two reasons.

First, the study clearly demonstrates that the neural network underlying future thought is not isolated in the brain's frontal cortex, as some have speculated. Although the frontal lobes play a well-documented role in carrying out future-oriented executive operations, such as anticipation, planning and monitoring, the spark for these activities may well be the very process of envisioning oneself in a specific future event, an activity based within and reliant upon the same neurally distributed network used to retrieve autobiographical memories.

Second, within this neural network, patterns of activity suggest that the visual and spatial context for our imagined future often is pieced together using our past experiences, including memories of specific body movements and visual perspective changes � data stored as we navigated through similar settings in the past.

These findings, McDermott suggests, offer strong support for a relatively recent theory of memory, which posits that remembering the past and envisioning the future draw upon many of the same neural mechanisms. Previous speculation has been based largely on the anecdotal observation of very young children, cases of severe depression and brain damaged persons with amnesia.

"There's a little known and not that well investigated finding that if you have an amnesic person who can't remember the past, they're also not at all good about thinking about what they might be doing tomorrow or envisioning any kind of personal future," McDermott explains. They comprehend time and can consider the future in the abstract sense (e.g., that global warming is a concern for the future), but they cannot vividly envision themselves in a specific future scenario.

"The same is true with very small children -- they don't remember particularly what happened last month and they can't really tell you much of anything about what they envision happening next week. This is also the case with suicidally depressed people. So, there's this theory that it all goes hand-in-hand, but nobody has looked closely enough to explain exactly how or why this occurs."

In this study, researchers relied on functional magnetic resonance imaging (fMRI) to capture patterns of brain activation as college students were given 10 seconds to develop a vivid mental image of themselves or a famous celebrity participating in a range of common life experiences.

Presented with a series of memory cues, such as getting lost, spending time with a friend or attending a birthday party, participants were asked to recall a related event from their own past; to envision themselves experiencing such an event in their future life; or, to picture a famous celebrity -- specifically former U.S. President Bill Clinton -- participating in such an event.

The "Clinton-Imagine" task was introduced to help researchers establish a baseline level of brain activity for a cognitive event that was in many ways similar to the other two tasks but did not involve the mental projection of oneself through time. Bill Clinton was chosen because pre-testing showed he was easy for participants to visualize in a variety of situations.

Comparing images of brain activity in response to the "self-remember" and "self-future" event cues, researchers found a surprisingly complete overlap among regions of the brain used for remembering the past and those used for envisioning the future � every region involved in recollecting the past was also used in envisioning the future.

During the experiment, participants were not required to describe details or explain the origin of mental images elicited by the memory cues, but in post-testing questionnaires most indicated that they tended to place future-oriented images in the context of familiar places (e.g. home, school) and familiar people (e.g. family, friends), which would require the reactivation of those images from neural networks responsible for the storage and retrieval of autobiographical memories.

Conversely, the neural networks associated with personal mental time travel showed significantly less activity when participants imagined scenarios involving Bill Clinton. The reason, researchers suggest, is that participants had no personal memories of direct interaction with Clinton, and thus, any images of him had to be derived from neural networks responsible for semantic memory � our context-free general knowledge of the world. In fact, participants later reported that their mental images of Clinton tended to be less vivid (e.g. "I see Bill Clinton at a party in the White House, alongside several faceless senators").

"Results of this study offer a tentative answer to a longstanding question regarding the evolutionary usefulness of memory," McDermott concludes. "It may just be that the reason we can recollect our past in vivid detail is that this set of processes is important for being able to envision ourselves in future scenarios. This ability to envision the future has clear and compelling adaptive significance."

Post to:

Labels: brain, future_vision, imaging, mind_games, vision