BCI Potential Currently Unrealized
The ability to control an external device via thought has the potential to greatly enhance humanity. Applications nearing commercial viability include neuroprosthetics, spelling and communication, as well as control of phones, computers, drones and VR worlds. Despite great progress however much work remains. BCIs are not yet a viable commercial technology (Chaudhary et. al., 2021). They are scarcely used outside laboratories for practical applications, and effective translation from proof-of-concept prototypes into reliable applications remains elusive. (Chavarriaga et. al., 2016).
The Mind is Critical
Understanding the mind is a central component of brain signal decoding. Categorizing the user’s mental states and processes accurately allows a correspondingly accurate decoding (and encoding) of the user’s brain signal. What does the user’s brain signal mean, exactly, from a subjective perspective? What task or mental command does it represent?
Meaning is a subjective phenomenon. It is based 100% on conceptual understanding — i.e. the mind. The problem is the mind is currently relatively ignored. During a BCI task only one (ex: “imagine the cursor moving to the left”) or maybe two (ex: “imagine… with high cognitive workload”) mental states or processes are considered.
The set of mind categories can and should be expanded. Whatever mind labels are used enable corresponding brain signals to be decoded — as such. And the more comprehensive the set of categories the better. A signal decoded as “raise left arm” is not as useful for decoding as that + associated components of mind: such as “to grasp my phone, to send an important text with focused attention, while feeling excited and confident, predicting the feeling of the phone against my hand, and the inner speech ‘grasp phone’ phoneme sequence.”
Decoding in turn is central to BCI. For a brain signal to trigger (after feature extraction and processing) an external device command it must be decoded accurately.
Classifiers and the Mind
A narrow view of the user’s mind is reflected by an equally narrow set of classifiers. If your classifier, or classifier set, represents a single mental state or process (“paradigm”) the neural correlate of that mind feature can only be decoded as that feature. A classifier can only decode/classify signal features that are labeled, and no others. Or at least, explicit and knowing classification is limited to that mental component.
However with most tasks, especially in the real world, dozens of mental states and processes are active; many simultaneously. With the above cursor task for example, the user’s mind might include not just imagination but perception (visual, auditory, somatosensation…), emotion (excitement, confidence, frustration, anxiety), belief, motivation, goals, attention, cognitive workload, intention, and prediction as well as hunger, thirst, fatigue, pain and much more.
If the categories of mind, and corresponding classifier set, can be broadened to include a much higher % of the mind, then this is great news for signal decoding. The brain’s activity can now be more comprehensively and precisely covered by this set of classifiers. This of course assumes a 1-to-1 correspondence between mind and brain. Yet is this the case?
The Mind/Brain Connection
I argue mind and brain activity mirror one another. Human movement demonstrates how extremely close this relationship is. The efferent signal is responsive (directly or indirectly) to any mental state or process. This includes not only the motor cortex signal. The efferent signal also responds to paths leading to the motor cortex (an intention such as “raise my right arm”), and to paths associated with it (perception, recognition, meaning, emotion, prediction, higher goals, level of attention…). If brain activity does not mirror mind activity, then how could a person control his/her movement with skill and precision, in real time? How could perception, thought, and intention affect movement — and do so instantly?
More generally, the idea mind and brain are closely connected is the entire basis of the field of cognitive neuroscience. It’s been well established that a particular type or instance of stimulus, thought, emotion, imagination, intention and the rest are closely connected to (large scale, coordinated) patterns of neural activity. Distinct brain oscillations underlie specific cognitive functions (Cox et. al., 2018).
The good news is the mind’s potential to unlock the secrets of the brain (or clarify existing knowledge) is right at our fingertips. The first step is to take it seriously as a “real” phenomenon. Then understand it — well enough to define its contents, and their function through space and time. In other words create a model of the mind. Using an accurate one, a mental state and process set can be defined, weighted, and connected. This “functional mind map” can then be connected to brain activity and signal. From here, a mind/brain signal “signature” can be developed. This can be used as the basis of a classifier set, to decode and classify matching brain signals.
The User-Centered Approach
Taking advantage of the user’s subjective mind in BCI is supported by other researchers. They argue the field would benefit greatly from a more “user-centered” approach. The user’s role in device control and performance has been minimized, and taking into account his or her mental states and skills could have a substantial impact in improving BCI efficiency, effectiveness and usability (Lotte, Jeunet, Mladenovic et al., 2018).
I not only agree, but argue the user-centered approach should be extended further — beyond user states and traits to include the entire range of mental states and processes. Instead of focusing on a few ad hoc mental states, there is an opportunity to systematically define the user’s mind. The only thing required is a mind model: its states and processes as they occur through space and time, inside the brain. Once defined, this set can then be narrowed to the components most strongly and consistently activated during device use, within a given context — environment, situation, activity, task, recent performance etc.
Mind-based Brain Signal Classifiers
As a mind component, or set of components, and corresponding brain signal is expressed across task trials, a signature of activity can be identified. This is a range of similar (mind, and brain signal) expression. This signature can then be used as, or to develop, a brain signal classifier.
Developing a mind-based classifier involves the following: (1) acknowledge the mind exists (the mind is not only “the brain” but a subjective phenomenon in its own right), (2) define it using a mind model, (3) define the mind’s components active during a mental command, within a mind/brain/body/environment context, (4) identify the brain signal characteristics corresponding to these mind signatures, to create brain signal signatures, and classifiers, and (5) identify those most interesting, motivating and relevant to the user, that he or she can activate with strength & consistency.
A set of mind-based classifiers can be developed that correspond to the user’s combination of mind components at play. Having potentially dozens of viable classifiers at one’s disposal greatly increases the classification “shots on goal” possible. And, this set can be selectively applied (or selectively weighted) depending upon the user’s predicted mind activation, during a given task + context.
Standing in the way of a mind-first classifier development approach however is the lack of a mind model. The human mind is poorly understood (Poldrack & Yarkoni, 2016). Currently no accurate or systematic method of defining a mental state or process, within a given task & context, exists. Because of this shortcoming even a sharp turn toward a user-centered approach, though helpful, will have limited effect.
User Empowerment
Mind-based classifiers, and a greater emphasis on the subjective mind generally, can empower the BCI user. Focusing on the mind as one side of the mind/brain coin allows the user to see she’s in control. As she manipulates her own mind, she simultaneously controls her brain signal. This can be done with clear conscious intent. However the mind is manipulated, the brain follows. Clarifying this relationship helps the user control her brain signal more naturally yet intentionally.
Classifiers based on the mind also encouraged their personalization. In consultation with others, the user can research and define the states & processes easy for her to achieve, strongly and consistently. These could include aspects of her mind most strongly aligned with her lifestyle, interests, favorite activities, professional goals and so on.
In addition, mind-first classifiers can serve as “mind targets” for the user to try to “hit” or activate. More precise and comprehensive targets — clearly understood by the user — cannot help but increase user performance and classification accuracy.
The Core Problem: The Mind is Largely Ignored
The idea the human mind strongly connects to brain activity & signal is obvious. After all it’s the basis of the entire field of cognitive neuroscience. Yet there’s a tendency (driven by materialism) to minimize or ignore the subjective within brain science. This neglect trickles down to the field of BCI. Although a specific (mental or behavioral) task may be described in great detail, the mind that executes it — beyond that singular mental component — is often ignored.
Minimizing the mind causes it, and corresponding brain signal, to be defined (encoded and decoded) far from optimally. Signal classification suffer accordingly. Many positive states of mind — feeling calm, confident, happy, content, motivated etc. remain unacknowledged. So do unwanted states — feeling frustrated, impatient, anxious, distracted, in pain etc. If these mental categories are not defined or accounted for subjectively, they won’t be included as part of the classifier, and won’t be classified (at least not knowingly).
The good news is any mental phenomena can be included as part of a classifier, wanted or unwanted. If desired and consistently activated, a state can be represented by that classifier. Or if sporadically and unpredictably activated, a “noise classifier” could be matched to this state; with subsequent matching signals filtered out. Any component of mind can be labeled as noise or as part of the signal — once acknowledged as part of the mind/brain system in the first place.
Summary
Once the user’s mind is defined accurately, connected to the brain, and used to build brain signal signatures, higher-performing classifiers and “mind targets” for the BCI designer, trainer and user to benefit from can be developed.
References
Chaudhary, U., Chander, B. S., Ohry, A., Jaramillo-Gonzalez, A., Lule, D., Birbaumer, N. (2021). Brain Computer Interfaces for Assisted Communication in Paralysis and Quality of Life. International Journal of Neural Systems v. 31. https://doi.org/10.1142/S0129065721300035
Chavarriaga, R., Fried, O., Kleih, S., Lotte, F., Scherer, R. (2016). Heading for new shores! Overcoming pitfalls in bci design. Brain-Computer Interfaces, 4,60.
Cox, R., Schapiro, A., Stickgold, R. (2018). Variability and stability of large-scale cortical oscillation patterns. Network Neuroscience, 2(4),481. doi: 10.1162/netn_a_00046
Lotte, F., Jeunet, C., Mladenovic, J., N’Kaoua, B., Pillette, L. (2018). A BCI challenge for the signal processing community: considering the user in the loop. Signal Processing and Machine Learning for Brain-Machine Interfaces, IETpp.1-2
Poldrack, R.A., Yarkoni, T. (2016). From brain maps to cognitive ontologies: informatics and the search for mental structure. Annual Review of Psychology, 67, 587.