The field of virtual reality (VR) and augmented reality (AR) is continuously evolving, with significant advancements on the horizon. The current emphasis is on the shift from physical controllers to the utilization of hand and eye tracking for inputs. However, there exist even more profound prospects on the horizon, particularly within the domain of neural interfaces.

OpenBCI, a company situated in Brooklyn, stands at the vanguard of developing tools for noninvasive brain-computer interface (BCI) technology. They have taken their expertise in sensory systems and integrated them into a mixed-reality headset named Galea, a brain connected AR-VR headset. We expect this exciting headset to release later this year, offering users an innovative and immersive experience.

An Innovative Sensor Platform for Virtual Reality: Brain Connected AR-VR Headset

Galea, developed by OpenBCI, incorporates a wide range of sensors, creating a comprehensive solution. These sensors include EEG (electroencephalography), EMG (electromyography), EDA (electrodermal activity), PPG (photoplethysmography), and eye tracking.

EEG sensors measure the electrical activity of brain signals. OpenBCI employs rubbery-tipped sensors that people can near the scalp. These electrodes are effective even when dry. But it is important to ensure minimal interference from hair to obtain optimal signal quality.

EMG sensors, on the other hand, are responsible for measuring nerve and muscle electrical activity. In Galea, these sensors are integrated into a facemask on the headset, positioned around the forehead, eyes, and cheeks.

Galea’s sensors capture even the slightest movements of my facial muscles and translate them into measurable readings. Note that this is unlike VR headsets like the Quest Pro. The latter relies on face cameras to detect specific physical movements. Note that Galea’s readings are entirely electrical, this means that it has the potential to detect incredibly subtle movements that resemble simple neural impulses.

Image By- Scott Stein/CNET

Meta, another company, is also working on EMG technology for wristbands. They are making wristbands that people can use in conjunction with future headsets. However, this wrist technology focuses solely on measuring finger and hand movements through the wrist. In contrast, OpenBCI’s sensors are specifically to monitor facial movements.

Additionally, Galea incorporates EDA (electrodermal activity) sensors, which measure the electrical signals produced by sweat on the skin. This provides an additional layer of information for a more comprehensive understanding of the user’s physiological responses.

Other Things You Need to Know

The EDA sensors integrated into Fitbit’s Sense smartwatch are primarily used for measuring stress levels. OpenBCI has similarly incorporated EDA sensors into the forehead section of the Galea headset, offering stress-sensing capabilities as well.

PPG, which stands for photoplethysmography, is a method of optically sensing heart rate, commonly found in most smartwatches. In the final version of Galea, PPG measurements of the forehead area also play a role. The sensor array of Galea combines with an existing VR-AR headset, such as the Varjo XR-3. This is also true for the more affordable Varjo Aero. To operate the system, you need to connect it to a computer that runs the software and performs data analysis.

OpenBCI’s sensor array benefits from the advanced features of Varjo’s high-resolution display and passthrough video mixed reality. This also provides a multitude of software possibilities in VR and AR scenarios. However, it’s important to note that OpenBCI’s sensors are versatile. So they can function independently of a VR headset.

The Vision Pro developed by Apple is a promising platform for OpenBCI. This is mainly due to its impressive processing power and standalone capabilities. Conor Russomanno, the CEO, and co-founder of OpenBCI, acknowledges the potential of collaborating with platforms like Vision Pro or future AR and VR platforms. He also draws parallels between Apple’s recent emphasis on the computer aspects of mixed reality and OpenBCI’s perspective on the available opportunities in this field.

Accessibility goals: Brain Connected AR-VR Headset

Image By- Scott Stein/CNET

OpenBCI’s sensory array possesses the ability to explore various possibilities simultaneously. Instead of focusing on a singular objective, the system’s sensors have the potential to facilitate research endeavors while also enabling interactions with computers. A notable collaboration involved Christian Beyerlein, a hacker living with spinal muscular atrophy, who utilized OpenBCI’s sensory array to control a drone using facial muscle impulses. This remarkable demonstration, presented as a TED talk, exemplified how brain-computer interfaces can revolutionize accessibility and provide enhanced control over virtual and real-world technologies.

The purpose of EMG technology is to detect extremely subtle electrical impulses. We are talking about precision to the point where no visible muscle movements are apparent. However, achieving seamless integration between sensors, algorithms, and human input at such a level may require considerable refinement over time. OpenBCI’s diverse array of sensors has the potential to generate a wealth of data that could inform future research directions and inspire the development of novel interfaces. 

Furthermore, these sensors can offer insights into how the utilization of VR and AR impacts the brain and attention. Previous endeavors to study cognitive processes using sensors on VR headsets, such as the HP Omnicept with its heart-rate sensor or eye-tracking-enabled headsets, have already laid the groundwork in this domain.

A sensory platform with the potential for broader applications beyond headsets

Indeed, Galea, developed by OpenBCI, functions as both a VR and AR headset. While its compatibility with Varjo’s hardware is a significant aspect, it’s essential to note that the sensor array of Galea can also be utilized independently. This aspect is particularly intriguing when considering the potential of a future where wearable devices interact with one another, enhancing our everyday interactions. 

Although this vision of advanced sensor-driven wearables is still a distant prospect, the initial steps toward such a future seem to be emerging through the sensor technologies integrated into Galea by OpenBCI. Presently, the task of convincing individuals about the worth of VR, AR, and wearable visual technologies remains challenging. 

However, enhancing our interfaces with spatial computing and the physical world could potentially hold the key to the evolution of VR/AR into something profoundly significant, albeit potentially unsettling. The ongoing development of personal technology indicates a growing connection with our senses and brains, but the glimpses we have witnessed so far only scratch the surface of what is possible.