Using AI, we are one step closer to translating brain activity into spoken words
We use our voices to connect to others—to family, friends, even a stranger you bump into on the street. It’s how we communicate, and express our thoughts and feelings. So when people lose access to their voice, it’s devastating, says Nima Mesgarani, a neuroscientist at Columbia University's Mortimer B. Zuckerman Mind Brain Behavior Institute. This might happen because of conditions like ALS, a stroke, or an injury to the vocal cords.
In a new study in Scientific Reports, Mesgarani and his colleagues showed that it might be possible to translate someone’s inner thoughts into speech, like a kind of mind reading. When we listen to others talking, our brains process the sound and make meaning of it, causing specific activity in our brains. But it hasn’t been simple to just take that information from the brain and turn it into speech, Mesgarani tells me.
When he and other researchers have tried to reconstruct speech from brain activity before, they “were able to produce audio that sounded kind of similar to the original speech, but not intelligible in any way,” he says.
In the new work, they used artificial intelligence with advanced speech processing capabilities to reconstruct sounds that are much more understandable than ever before. “This is a huge milestone, and we weren’t sure we could reach it,” he says.
To do it, they asked people with epilepsy, who were already having brain surgery, to listen to sentences from a children's book, "Hank the Cowdog,” and recorded what was going on in their brains. That brain activity acted like a translation guide, and helped train an artificial intelligence algorithm, called vocoder, to decode the “thoughts.”
Then, they asked the same people to listen to numbers being spoken out loud, while the vocoder monitored their brains. It spoke the numbers back to them in “a robotic-sounding voice,” a press release says. (You can hear what the robot voice sounded like in the Supplementary Information section of the paper, which is open access.)
“Our ultimate goal is to develop technologies that can decode the internal voice of a patient who is unable to speak, such that it can be understood by any listener,” Mesgarani says. “One of the biggest barriers has been the indelibility of the reconstructed sound. Our algorithm is the first to generate a sound that is actually intelligible to human listeners, therefore bringing this concept a step closer to reality.”
Chickens might one day lay eggs containing important ingredients for medication
A lot of medications are based on proteins—large molecules made of amino acids that can do different things in our bodies depending on their shape. But making these proteins can be tough and expensive. It involves getting a living cell—like one from a bacteria, yeast or mammal—to produce the protein, and usually only mammal cells can make the more complex shapes.
So, what if a chicken could lay an egg with a desired protein already inside? A new paper in BMC Biotechnology shows that it’s possible to make proteins this way, and it could be an appealing approach to making drugs in the future.
We already use eggs to grow viruses that are used in vaccines, like the flu shot—you take a fertilized egg and inject a virus into it, which then replicates inside. Lisa Herron, a postdoctoral research fellow at the University of Edinburgh’s Roslin Institute and the head of the Avian Biopharming Business Unit, tells me that their new approach is a little different.
They permanently inserted a gene into the DNA of chickens, which gave their bodies the instructions to make a protein they wanted. They also included a piece of DNA called a “promoter,” which told the protein where and when to be made.
“That means we can control the expression of the protein so it only expresses in egg white of laying hens,” Herron says. Since the gene with the instructions is put into the DNA of the chickens, any baby chicken they make will also carry this gene. The chickens are none the wiser; they just lay their eggs as normal.
To get the protein out of the egg white, they use a process called chromatography which separates proteins based on different traits, like size. I ask her if, after the protein is extracted, the egg could be scrambled and eaten. But there won’t be any cross-contamination of these eggs and your breakfast: ”As these animals are GM, neither the eggs nor the chickens go into the food chain, and are kept strictly separated from food chain animals and eggs,” she tells me.
In the new research, they made chickens lay eggs with two different proteins, a human protein that has powerful antiviral and anti-cancer effects, and human and pig versions of a protein that’s being developed into a drug to repair damaged tissues. Just three eggs made enough of a drug to be “clinically relevant,” a press release says. And since chickens lay up to 300 eggs per year, it could be a really cost-effective way to make some medicines.
“Chickens will never replace cell-based systems,” Herron tells me. “But our hope is that for the types of proteins where the chicken may be a superior method, we and hopefully other drug makers will use it.”
Being a morning person might be better for your health
Each of us has in internal body clock that lets our body know if it’s day or night. But all of our clocks aren’t wound exactly the same way. Some people are naturally larks (who get up and go to bed early) or owls (who get up and go to bed late).
If you are a lark or an owl is partially driven by differences in our DNA—but it’s still fuzzy exactly how your unique circadian clock might influence your health.“There have been many reported links between morning preference and lower risk of obesity, diabetes, depression, and schizophrenia,” says Jacqueline Lane, a postdoctoral fellow at the Broad Institute.
To learn what impact a person’s circadian clock has on different diseases, we need really large studies—and a big one just came out in Nature Communications. Previous studies have been a decent size, looking at 128,286 people’s DNA and finding 24 gene variations that are associated with a lark-ness or owl-ness. In the new study, Lane and her colleagues looked at genomic information from around 250,000 people in the US using 23andMe data, and another 450,000 people from the UK, from a UK Biobank study—a whopping total of 697,828 people overall.
All the subjects were asked if they were morning or evening people, and then their DNA was examined to see if morning and evening people had any genes in common that had to do with sleeping patterns. To confirm what people reported, they also had more than 85,000 people wear activity trackers on their wrists to see what time people got up in the mornings and went to sleep.
They found that there are potentially several genetic factors that influence sleep timing. Some lead to subtle changes in the speed of the body clock, and others they saw led to differences in the retina, in the eye. “The body clock cycle is slightly longer than the 24-hour daily cycle,” a press release says. “The eye tissue connection may help explain how the brain detects light to ‘reset’ the body clock each day and to align with the 24-hour cycle.”
Surprisingly, Lane tells me that they did not find a causal link between being a morning person and obesity or diabetes risk, as had been previously reported. But they did find a potential causal link between being a lark and having a lower risk for schizophrenia.
If you’re an owl who wants to be a lark, can you just switch? It might be more complicated than that. “If someone is a night owl, that is shaped by genetics along with age, sex, and environmental factors,” Lane tells me. “Previous research suggests trying to change a night owl into a morning lark may have negative health consequences, but there are likely individual differences in how easily we can adapt and the consequences of that adaptation.”
For more on larks and owls, and all things circadian, check out when I asked experts every question I could think of about sleep.
Your weekly health and science reads
What time is the Super Bowl? We asked theoretical physicist Carlo Rovelli. By Gary Gramling in Sports Illustrated.
“What” “time” “is” the Super Bowl?
A body is a bill to pay. By Laura Eppinger in The Rumpus.
A powerful and personal look at the ramifications of student debt on the body and mind.
Inside the experiment that could end infertility. By Brooke Borel on Medium.
“In the future, we may be making eggs and sperm from our skin. A lot has to happen first.”
Personalized health advice every 60 seconds. By Richard Sprague in Neo.Life.
But this one product for near-constant information about what’s going on in your body.
The 500-year-long science experiment. By Sarah Zhang in The Atlantic.
How’s this for optimism about human civilization: These microbiologists started an experiment that won’t be done until 2514.
Sign up for our newsletter to get the best of Tonic delivered to your inbox.
This article originally appeared on VICE US.