Multi-modal Omni-directional 3D Sensor for Behavioral Analysis of People

Project duration	Funding
08/2016 - 07/2019	BMBF
Project head	Staff
Prof. Dr. rer. nat. Andreas Wendemuth	M.Sc. Juliane Höbel-Müller, Dr.-Ing. Ronald Böck

Project description

Motivation and project's aim

In the Alliance 3Dsensation machines are enabled to visually capture and interpret complex scenarios by applying innovative 3D technologies. Thus, machines are becoming situational partners and personalized assistants. Thanks to the new form of human-machine interaction, 3Dsensation creates access to living and working environments, which are regardless of age and physical performance.

Motivated by the demographic change and the associated societal challenges, an intelligent sensor for the domestic assistance of the elderly is to be developed for the healthcare field in the MOVA3D project. To fully perceive a domestic environment with a single sensor, a novel omnidirectional 3D optical measurement principle is combined with the acoustic capture of an environment for multimodal information gathering. Highly accurate (3D) video and audio data are the prerequisites for the subsequent recognition of complex human activities in everyday situations and interactions with technical systems, as well as the identification of relevant deviations. This automatic human behavior analysis is the basis for corresponding assistance functions as well as extensive interaction via audio and light-based interfaces. User studies regarding acceptance and functionality issues are essential parts of the MOVA3D project. In addition to the integration into current AAL and home automation systems, a cross-field application is conceivable and desired.

Within the project the OvGU is responsible for the development of emotion and disposition recognition in speech. Additionally, acoustic events in domestic environments are to be classified. By using audio and video information, a multimodal user behavior model will be created for estimating user’s mental and physical health.

Approach

The technical development is based on a requirements definition regarding a user study covering the prospective users' needs. During the study, specific daily routines of representative users and their behavioral patterns in their specific domestic environment are recorded, analyzed and documented using surveys. The study is carried out with the help of a mock-up, which simulates the intended functionality of the sensor system as realistic as possible by using technology manually used, cameras, speaker, etc. and design. So, the mock-up will reproduce representatively the reaction and behavior of users.

The findings from this analysis on a sample basis are transferred into relevant use cases and personas. These serve as the basis for a user-centered and target-group-specific technical development, which will ensure acceptance of the sensor system by the user. The results are recorded in requirement documents.

In the final phase of the project, validation and evaluation of the multimodal, omnidirectional sensor will take place with regard to acceptance issues. The results will give implications regarding a possible re-design of the prototype.

INSTITUTE FOR

INFORMATION TECHNOLOGY AND

COMMUNICATIONS