cross pond high tech

You don’t have to think about it: when you speak, your brain sends signals to your lips, tongue, jaw, and larynx, which work together to produce the intended sounds.

Now scientists in San Francisco say they’ve tapped these brain signals to create a device capable of spitting out complete phrases, like “Don’t do Charlie’s dirty dishes” and “Critical equipment needs proper maintenance.”

The research is a step toward a system that would be able to help severely paralyzed people speak—and, maybe one day, consumer gadgets that let anyone send a text straight from the brain. 

A team led by neurosurgeon Edward Chang at the University of California, San Francisco, recorded from the brains of five people with epilepsy, who were already undergoing brain surgery, as they spoke from a list of 100 phrases.

When Chang’s team subsequently fed the signals to a computer model of the human vocal system, it generated synthesized speech that was about half intelligible.

The effort doesn’t pick up on abstract thought, but instead listens for nerves firing as they tell your vocal organs to move. Previously, researchers have used such motor signals from other parts of the brain to control robotic arms.

“We are tapping into the parts of the brain that control these movements—we are trying to decode movements, rather than speech directly,” says Chang.

In Chang’s experiment, the signals were recorded using a flexible pad of electrodes called an electrocorticography array, or ECoG, that rests on the brain’s surface.

To test how well the signals could be used to re-create what the patients had said, the researchers played the synthesized results to people hired on Mechanical Turk, a crowdsourcing site, who tried to transcribe them using a pool of possible words.  Those listeners could understand about 50 to 70% of the words, on average.

“This is probably the best work being done in BCI [brain-computer interfaces] right now,” says Andrew Schwartz, a researcher on such technologies at the University of Pittsburgh. He says if researchers were to put probes within the brain tissue, not just overlying the brain, the accuracy could be far greater.

Previous efforts have sought to reconstruct words or word sounds from brain signals. In January of this year, for example, researchers at Columbia University measured signals in the auditory part of the brain as subjects heard someone else speak the numbers 0 to 9. They were then able to determine what number had been heard.

Brain-computer interfaces are not yet advanced enough, nor simple enough, to assist people who are paralyzed, although that an objective of scientists.

Last year, another researcher at UCSF began recruiting people with ALS, or Lou Gehrig’s disease, to receive ECoG implants. That study will attempt to synthesize speech, according to a description of the trial, as well as asking patients to control an exoskeleton supporting their arms.

Chang says his own system is not being tested in patients. And it remains unclear if it would work for people unable to move their mouth. The UCSF team says that their set-up didn’t work nearly as well when they asked speakers to silently mouth words instead of saying them aloud.

Some Silicon Valley companies have said they hope to develop commercial thought-to-text brain readers. One of them, Facebook, says it is funding related research at UCSF “to demonstrate the first silent speech interface capable of typing 100 words per minute,” according to a spokesperson.

Facebook didn’t pay for the current study and UCSF declined to described what further research it's doing on behalf of the social media giant. But Facebook says it sees the implanted system is a step towards the type of consumer device it wants to create.

“This goal is well aligned with UCSF's mission to develop an implantable communications prosthesis for people who cannot speak – a mission we support. Facebook is not developing products that require implantable devices, but the research at UCSF may inform research into non-invasive technologies,” the company said.

Chang says he is “not aware” of any technology able to work from outside the brain, where the signals mix together and become difficult to read.

“The study that we did was involving people having neurosurgery. We are really not aware of currently available noninvasive technology that could allow you to do this from outside the head,” he says. “Believe me, if it did exist it would have profound medical applications.”

SpaceX and Tesla CEO Elon Musk is backing a brain-computer interface venture called Neuralink, according to The Wall Street Journal. The company, which is still in the earliest stages of existence and has no public presence whatsoever, is centered on creating devices that can be implanted in the human brain, with the eventual purpose of helping human beings merge with software and keep pace with advancements in artificial intelligence. These enhancements could improve memory or allow for more direct interfacing with computing devices.

Musk has hinted at the existence of Neuralink a few times over the last six months or so. More recently, Musk told a crowd in Dubai, “Over time I think we will probably see a closer merger of biological intelligence and digital intelligence.” He added that “it's mostly about the bandwidth, the speed of the connection between your brain and the digital version of yourself, particularly output." On Twitter, Musk has responded to inquiring fans about his progress on a so-called “neural lace,” which is sci-fi shorthand for a brain-computer interface humans could use to improve themselves.

These types of brain-computer interfaces exist today only in science fiction. In the medical realm, electrode arrays and other implants have been used to help ameliorate the effects of Parkinson’s, epilepsy, and other neurodegenerative diseases. However, very few people on the planet have complex implants placed inside their skulls, while the number of patients with very basic stimulating devices number only in the tens of thousands. This is partly because it is incredibly dangerous and invasive to operate on the human brain, and only those who have exhausted every other medical option choose to undergo such surgery as a last resort.

This has not stopped a surge in Silicon Valley interest from tech industry futurists who are interested in accelerating the advancement of these types of far-off ideas. Kernel, a startup created by Braintree co-founder Bryan Johnson, is also trying to enhance human cognition. With more than $100 million of Johnson’s own money — the entrepreneur sold Braintree to PayPal for around $800 million in 2013 — Kernel and its growing team of neuroscientists and software engineers are working toward reversing the effects of neurodegenerative diseases and, eventually, making our brains faster and smarter and more wired.

Elon Musk’s Neuralink, the secretive company developing brain-machine interfaces, showed off some of the technology it has been developing to the public for the first time. The goal is to eventually begin implanting devices in paralyzed humans, allowing them to control phones or computers.

The first big advance is flexible “threads,” which are less likely to damage the brain than the materials currently used in brain-machine interfaces. These threads also create the possibility of transferring a higher volume of data, according to a white paper credited to “Elon Musk & Neuralink.” The abstract notes that the system could include “as many as 3,072 electrodes per array distributed across 96 threads.”

The threads are 4 to 6 μm in width, which makes them considerably thinner than a human hair. In addition to developing the threads, Neuralink’s other big advance is a machine that automatically embeds them.

Musk gave a big presentation of Neuralink’s research Tuesday night, though he said that it wasn’t simply for hype. “The main reason for doing this presentation is recruiting,” Musk said, asking people to go apply to work there. Max Hodak, president of Neuralink, also came on stage and admitted that he wasn’t originally sure “this technology was a good idea,” but that Musk convinced him it would be possible.

In the future, scientists from Neuralink hope to use a laser beam to get through the skull, rather than drilling holes, they said in interviews with The New York Times. Early experiments will be done with neuroscientists at Stanford University, according to that report. “We hope to have this in a human patient by the end of next year,” Musk said.

During a Q&A at the end of the presentation, Musk revealed results that the rest of the team hadn’t realized he would: “A monkey has been able to control a computer with its brain.”