The decoder reads the thoughts recorded by a brain scanner | Sciences
Three subjects were made to hear a Podcast affiliate The New York Times and monologues from a popular Anglo-Saxon show while their brains were scanned. Using a decoder designed by scientists, American scientists were able to convert brain scan graphs not just into complete sentences, but into texts that reproduced with great accuracy what they heard. According to their results, published today in the scientific journal Natural neuroscienceThis decoder called semantic was also able to express what they were thinking and what was going through their heads while they were watching silent movies.
Since the beginning of the century, and especially in the last decade, great progress has been made in the design of brain-machine interfaces (BCIs). Most people unable to speak or even move all their muscles wanted to communicate. But most of these systems require opening the skull and placing an array of electrodes directly into the brain. Another less invasive technique is based on functional magnetic resonance imaging (fMRI). Here, the interface ends with a cap filled with electrodes placed on the head. This cap does not record direct neural activity, but changes in the level of oxygen in the blood that it causes. This poses resolution problems. On the one hand, due to access from the outside, and on the other hand, changes at this level occur at intervals of up to 10 seconds and in that period many words can be said.
To solve these problems, a group of researchers from the University of Texas (USA) relied on an artificial intelligence system that seems familiar to many: GPT, the same system on which the ChatGPT bot is based. This language model, developed by the OpenAI artificial intelligence lab, uses deep learning to generate text. In this investigation, they trained it on fMRI images of the brains of three people who were played 16 hours of audio from a doctor’s office. The New York Times and program Butterfly Clock Radio. In this way they were able to match what they saw to their representation in their heads. The idea is that when they hear another text again, the system can make predictions based on the patterns of what it has already learned.
This is the original GPT, not like the new one [ChatGPT se apoya en la última versión de GPT, la 4]. “We collected a lot of data and then built this model, which predicts brain responses to stories,” University of Texas neuroscientist Alexander Huth said in a webcast last week. Through this procedure, the decoder suggests the sequence of words “and for each of those words that we think might come next, we can measure how well the new sequence sounds and, ultimately, see if it matches the brain activity we are monitoring.” , the details.
This decoder has been called out, and rightly so. Previous interfaces recorded brain activity in motor regions that control the mechanical basis of speech, namely movements of the mouth, larynx, or tongue. “What they can decode is how a person is trying to move their mouth to say something. Our system works on a completely different level. Instead of looking at the low-level motor field, it works at the level of ideas, semantics, and meaning. That’s why they don’t register words.” the minute that someone heard or uttered it, but rather its meaning, ”explains Huth. For this, although the echoes recorded the activity of different brain regions, they focused more on those related to hearing and language.
Once the model was trained, the scientists tested it with six people who had to listen to texts different from those used to train the system. The machine decoded the fMRI images to approximate what the stories said. To ensure that the device worked at a semantic rather than a motor level, they repeated the experiments, but this time they asked the participants to imagine a story themselves and then write it down. And they found a great match between what was decoded by the machine and what was written by humans. Even more difficult, in the third installment, subjects had to watch scenes from silent films. Although the semantic decoder here fails more on specific words, it still captures the meaning of the scenes.
Neuroscientist Christian Herff leads research into brain-machine interfaces at Maastricht University (Netherlands) and nearly a decade ago created the ICB that allowed brainwaves to be converted into text, letter by letter. Herv, who was not involved with this new hardware, highlights the incorporation of the GPT language predictor. “This is really cool, since GPT inputs contain speech semantics, not phonetic or phonetic properties, as has happened in previous ICBs,” he says. “They show that a paradigm trained in what is heard can decode the semantics of silent films as well as imagined speech,” he adds. This scientist is “firmly convinced that semantic information will be used in brain-machine interfaces for speech in the future.”
“Their results are not applicable today, you need an MRI machine that takes up a hospital room. But what they achieved no one has achieved before.”
Arnaud Espinosa, a neurotechnologist at the Wyss Center Foundation in Switzerland
Arnau Espinosa, a neurotechnologist at the Wyss Center Foundation (Switzerland), published a paper last year on ICB with a very different approach that allowed an ALS patient to communicate. Regarding the current test, remember that “its results are not applicable today to the patient, you need the MRI equipment worth the millions that occupy a hospital room; but what they have achieved no one has achieved before.” The interface in which Spinoza enters was different. “We were going for a signal with less spatial resolution, but a lot more temporal resolution. We were able to figure out which neurons were being activated each millisecond, and then we were able to move on to phonetics and how to create a word,” he adds. For Espinosa, it will eventually be necessary to combine several systems, taking different cues. “Theoretically, it would be possible; more complicated, but possible.”
Rafael Yuste, a Spanish neuroscientist at Columbia University in New York (USA), has been warning for some time about the dangers posed by advances in his discipline. This research and a Facebook study demonstrate the possibility of decoding speech using a non-invasive neurotechnology. It is no longer science fiction,” he says in an email. These methods will have huge scientific, clinical, and commercial applications, but at the same time they portend the possibility of deciphering mental processes, because inner speech is often used for thinking. This is another argument for the urgent protection of mental privacy as a fundamental right human rights,” he adds.
Anticipating these concerns, the authors of the experiments wanted to see if they could use their system to read other people’s minds. Fortunately, they found that a model trained with one person was unable to decipher what another person heard or saw. Sure enough, they did one last series of tests. This time they asked the participants to count by sevens, think about and name the animals, or make up a story in their heads while listening to the stories. Here, the GPT-based interface, with all the technology that goes into the MRI machine and all the data that AI handles, fails more than a sideshow shotgun. For the authors, this would signify that mind-reading requires the owner’s cooperation. But they also caution that their research relied on the patterns of six people. Perhaps with dozens or hundreds of data, the danger, they admit, is real.
You can follow Theme in FacebookAnd Twitter And instagramor register here to receive Weekly newsletter.
