MAGI System - Mind Augmented Gesture Interaction

MAGI is a research project, part of a collaboration between the MISG group at EIA-FR and the DIVA group at UNIFR.


The interaction between Human-Environment is been always tied to the society technological level. With the explosion of electronics and computer science that habits are changed more rapidly than ever before. This new aspect takes the name of Human-Computer-Environment interaction. That has caused the rise of specialized branches of research analyzing new and more natural ways of control, communication and new methods to give to computers awareness about the ``context''.


In order to make human-computer interfaces truly natural, we need to develop technology that tracks human movement, body behavior and facial expression, and interprets these gestures in an affective way. The gesture-based interaction is a natural way to interact and can represent a substitution/complement for other forms of communications for impaired people or in special contexts.


The usual approaches to address the gesture recognition challenges go into the direction of computer vision and image processing. These methods are limited by some typical environmental constraints such as sensitivity to lighting changes, reliance on camera position, etc.; moreover, the images elaboration is very expensive in terms of computer processing power, making the real-time analysis a very difficult challenge.
On the other hand, the increasing centrality of the human being inside the design of new systems requires considering the most important of ``contexts'': the human being himself. In this direction psycho-physiological sensors such as Electroencephalogram (EEG), Electromyogram (EMG), Blood Volume Pressure (BVP) or Galvanic Skin Response (GSR) (just to mention the well-known technologies) give important information about the cognitive, affective or health subject's conditions.


In this work we interlace the concepts of Gesture Recognition, Context Awareness and Multimodal Interaction based on psycho-physiological signals in order to enhance our communication and control capabilities with the everyday object of our life, and overcome the obstacles present in the Computer Vision approach.​​