Windows
Linux
How and when did the idea to create NeuroChat come about? And why did you decide to work specifically with patients with speech disorders?
We were guided by the principle “help the sick, then you will come to the world of the healthy.” The brain-computer interface technology itself is about 50 years old. However, before us these were laboratory solutions or rather heavy hospital installations. Therefore, the idea arose to create a consumer-grade brain-computer interface in which you can use a regular computer at home, without the help of a special person. On the other hand, I wanted to give people who need it the opportunity to go online.
So, based on these two ideas, “NeuroChat” arose - a wireless neuro-headset that is placed on the user’s head and allows people with serious speech and movement disorders, for example, after a stroke, to communicate.
Tell us how to use NeuroChat step by step?
A computer screen is placed in front of the user, and a neuroheadset is placed on his head, which records weak bioelectrical activity of the brain from the visual cortex. A matrix of letters appears on the screen, and these letters begin to be highlighted one by one. When the row in which the letter the user needs is located is highlighted, then within 300 ms from the moment of illumination, the person’s brain will give a corresponding response. Then the letters in the horizontal row are highlighted and when the intersection of the horizontal and vertical illumination occurs, the machine, based on the classifier, deciphers the EEG recording and understands that this is the same letter that the user intended. The most important thing is that he must constantly think in his head about the desired letter, send a signal to his brain, and thus train the brain to respond to the intended symbols.
As soon as the computer realized that a given letter is at the intersection of two rows of backlight, it appears in the input line, and the user moves on to the next one. If he makes a mistake, there is an “erase” function. Again, the machine will offer ready-made words; you can also select them by concentrating mentally on the word.
This is not a mind reading technology, but a technology of intention to choose one or another symbol.
That is, formally it is possible to use pictograms or images?
Of course, we have had such experiments with our users who, for example, cannot read. Parents showed them images of certain food, objects, actions “drink”, “eat”, etc. In addition, “NeuroChat” has a Memory game, when you train your memory and at the same time learn to use the device. After all, its correct application requires concentration and attention. These are precisely the parameters that are very important for cognitive health, because in the information world in which we live, there is such an information flow that there is a serious problem of attention deficit disorder.
This neuroheadset, how convenient is it for an immobile person?
The neuroheadset is designed in such a way that a person can comfortably stay in it for up to 8 hours. We conducted experiments with different versions of the headset and users of different genders and ages in different rooms and positions: lying down, sitting, etc. To develop the neuroheadset, we attracted one of the most serious teams of industrial designers, led by Vladimir Pirozhkov from the high-tech prototyping engineering center Kinetics difficulty. It was important to take into account how comfortable it is for the user to be in the neuroheadset for a long time, whether it puts pressure on him, so the search was quite long and creative.
Are there such brain injuries that a person will not be able to use NeuroChat?
First of all, these are technological limitations. NeuroChat is based on the technology of evoked potentials that arise in the visual cortex. This means that a person must see. He may have difficulty seeing or be wearing glasses, but the main thing is to react to the images.
The second limitation is that the user must distinguish letters and be able to read. There are such violations, for example, again after a stroke, when a person forgets letters and how to form a word from them. And lastly, the user should not suffer from epilepsy. This is due to the fact that a person using NeuroChat constantly looks at the screen where letters and symbols flicker, which means we need to protect the user so that this backlight does not cause him any harm and does not cause convulsive readiness in the presence of an epileptic diagnosis.
The speed of typing using brain-computer interface technology without direct contact with the brain is still quite low. “NeuroChat” records the reaction to the illumination of the intended symbol, which occurs after approximately 300 ms. In addition, in order for the machine algorithm to recognize that a person has chosen exactly the right letter, it is necessary to record the evoked potential at least 3-4 times. Therefore, the speed of typing letters with this technology ranges from 6 to 10–12 characters per minute. A healthy person typing on a keyboard produces about 120 characters per minute, that is, approximately 2 letters per second.
Tiring, of course. Is there any way to speed up the set?
Of course, artificial intelligence is used, the algorithms of which can speed up typing due to predictive analysis (based on the frequency dictionary of the Russian language): for example, if a person chose the letters “p”, “r” and “i”, the algorithm will suggest choosing the word “Hello” ", etc. In addition, mathematics today allows you to create algorithms that will learn your vocabulary and suggest the most frequently used phrases. But then the equipment immediately becomes individual: it will be quite difficult for another person with different lexical preferences to use it. Whereas with the choice of words and phrases the story remains universal.
Does NeuroChat adapt to the user?
Yes, and each login to the system begins with calibration, since a person can be in a different physical and mental state, be tired, for example. We can wake up quite cheerful, or maybe, on the contrary, everything will fall out of our hands. Therefore, the speed of highlighting and word selection should correspond to the current state of the user.
Is it possible to use NeuroChat in a non-native language?
We even had a patient - a German teacher after a stroke. Yes, we have keyboard in Russian, German, English, French, Kazakh and Turkish. In addition, modern technologies make it possible to translate the entered text in real time. In 2019, we conducted a communication session between two patients: Denis from Russia and Deborah from the USA, each of them communicated in their own language, and the text was translated “along the way.” And since we are usually talking about simple sentences, it is not difficult to make the translation quite accurate. Therefore, virtually any translator that users need can be added to the chat.
Another thing is that adding new languages means creating a new matrix with letters. Therefore, for example, hieroglyphs will require multi-layer matrices due to the number of possible characters, but for Arabic there are no problems, the input direction simply changes.
Why was the choice not initially made in favor of neuroimplants when developing NeuroChat?
Of course, with invasive technology we are talking about a completely different interaction between the chip and the brain, a different level of efficiency, but also a different order of money and proof of effectiveness. The most important problem, which Elon Musk also faces in his company Neuralink, is rejection. With invasive implantation of electrodes, encapsulation may occur, and when it comes to a whole chip, the contact surface area is even higher. And then, by removing the chip, you can seriously damage something. Therefore, we are talking about serious research and financial resources.
It seems to me that if it is possible to do something non-invasively, then the world will look in the direction of such technologies. Firstly, this means that the person controls the device himself, and secondly, no outside intervention is required to remove or put on the headset.
That is, do you think that the world will look towards non-invasive technologies and, most likely, they will develop faster?
Of course, in my opinion, non-invasive technologies will develop faster, although I understand where this story is coming from with the popularity of invasive devices - cardiac stimulators that significantly improve people's lives. But here it is important to consider that we are talking about technology that can really improve the quality of life if it has suffered. It’s the same story with a chip for diabetics, which monitors glucose levels and injects insulin on time. The path of invasive technologies is, first of all, to preserve the quality of life.
29 February 2024
