For the successful integration of contactless interfaces for human-machine communication into everyday life, existing systems must be accessible to everyone without the need for lengthy training periods. As part of the M3S collaborative project with partners from research and industry (Bielefeld University; polyoptics GmbH and MediaBlix-IIT GmbH), Rhein-Waal University of Applied Sciences has developed a new hybrid, everyday-use, non-invasive brain-machine interface (BMI) for this purpose. It combines the measurement of brain signals and eye movements. Combining both signals in a hybrid system improves reliability and enhances the intuitive usability of the BMI, thereby also expanding its potential applications.
In particular, the system enables people with disabilities to participate in public and social life on their own terms. They can use it to communicate safely and quickly with other people, as well as, for example, to control household appliances independently. This independence from external help and care – which people without disabilities take for granted – can be a great asset in everyday life. Depending on the nature and severity of the user’s disability, the relative contribution of eye movements and brain signals (electroencephalography – EEG) can be individually adjusted. For example, in a condition known as locked-in syndrome, almost all eye muscles are paralysed. Depending on the severity, only vertical eye movements – or none at all – are possible. In such cases, the system focuses on measuring brain signals.
To this end, a particularly reliable form of brain signal (EEG) – known as Steady-State Visually Evoked Potentials (SSVEP) – was utilised. These signals occur whenever the retina is stimulated over a large area by a changing visual signal. If, for example, the gaze falls directly on a point on a monitor that is illuminated at a specific frequency, that exact frequency is clearly reflected in the EEG signals from the visual cortex. Based on the frequency measured in each instance, different commands can thus be encoded and an action selected accordingly. In the next step, the EEG signals measured in this way are analysed in real time by specially adapted algorithms. The algorithms were developed by the research group led by Professor Dr.-Ing. Ivan Volosyak, Professor of Biomedicine and Engineering and project leader.
To enable the display of as wide a range of frequencies as possible, a special monitor was developed during the course of the project by the Cognitive Engineering and Sensors Research Group at Bielefeld University, led by Professor Dr.-Ing. Ulrich Rückert, in collaboration with the company polyoptics from Kleve. Unlike a standard monitor, this device allows different refresh rates to be displayed simultaneously and independently of one another in various separate areas of the screen. When the viewer looks at these areas, SSVEP brain signals of different frequencies are generated, which can then be used to control various actions or devices.
To further enhance the reliability of the interface, the measurement of brain signals was combined with the measurement of eye movements (eye tracking). Combining both signals can reduce the selection speed and variance of the control options. To this end, the free, open-source eye-tracker developed by the company Mediablix-IIT in Bielefeld, under the leadership of Professor Dr Kai Essig, Professor of Human Factors and Interactive Systems, was integrated. In line with current open-source and open-access initiatives, the Libretracker software is freely available on the GitHub online platform. Open, patent-free solutions enable the development of cost-effective systems for a broad section of the population. Data security and privacy – which are particularly important in this context – can be better guaranteed in open-source solutions through inspection and analysis by independent experts. The innovative combination of EEG and eye-tracking therefore represents a new, particularly robust neurotechnology that can also be implemented cost-effectively.
The M³S research project is funded as part of the “NRW 2014–2020” programme by the European Regional Development Fund (ERDF) ‘Investments in Growth and Employment’ and the state government of North Rhine-Westphalia (reference number IT-1-2-001). LeitmarktAgentur.NRW, based at the Jülich Research Centre, is responsible for the funding programme.
