Computer Learns to Translate Brainwaves Into Sentences
By: Natalia Jones
Brainwaves are the way brain cells communicate with each other, and the result of that exchange are complex behaviors, such as emotions, movements and even language. In essence, brainwaves are the result of complex patterns in which a chain of neurons fire to elicit a specific behavior - an instruction manual for our body if you will. For decades, these neural instructions were a mystery, but a group of researchers from UCSF in San Francisco may have learned a way to eavesdrop on the hidden language of our brains with the use of machine learning.
In fact, the scientists managed to teach a machine to translate brainwaves they recorded from a number of participants into fully formed sentences with a less than 3% error rate - an unprecedented success compared to previous attempts.
The study detailing the procedure has been published recently in the scientific journal Nature Neuroscience, and it explains that researchers managed to train a computing system to decode neural patterns they recorded. The brainwaves were recorded by attaching around 250 electrodes to a specific location in the brain - the area of the brain cortex around the Sylvian fissure (see image below) - as 4 different participants spoke 30-50 various sentences.
These recordings were then fed into a computing system - a neural network. The system analyzed the brain patterns, and based on recurring features, it managed to decode these patterns into sentences. Interestingly, the machine managed to translate the brainwaves into text with a 97% success rate.
In addition, the researchers maintain that each brain signal is likely representative of a specific language unit - a word or a specific sound. Although the researchers didn't test if the system is capable of isolating separate words or interpreting new words and sentences, they do note that the performance of the machine became more accurate when they added practice sentences that didn’t end up in the final test into the system's database, which suggests that the decoder is likely capable of recognizing language units, possibly even words, and not just the entire sentence.
Image Source: Injurymap/ Wikimedia Commons (edited)
In addition, the researchers point out that, with each new added participant's data, the translations of the system improved and became more accurate, suggesting the machine should be capable of interpreting the neural activity of many people, and not just that of the 4 participants they tested.
The potential applications of a system capable of translating neuronal signals into human language are limitless. With the help of such a system, we may soon be able to give a clear voice to people with various language pathologies and neural disorders, develop a programming language of the human brain and finally teach programs how to translate sentences from one language to another without all the funny mistakes Google Translate makes all the time. Jokes and speculations aside, though, the possible applications of this technology are truly vast, and we shall follow its development with great interest.