External Funds Project (BMBF) FKZ: 16SV7209

Non-Irritating and Emotionally Sensitive Training System (IRESTRA)

Sub-Project: „Timing of Nonverbal and Emotional Expressions“                    

Duration: 2015 – 2017

Principal investigators: Dr. Uwe Altmann, Prof. Dr. B. Strauß

Research assistants: Dr. Magdalena Kaczmarek, Julia Mühleck, M. Sc.

The purpose of the joint project “IRESTRA” is the development of a computer-based system designed to assist with decentralised therapy tasks and interventions with a focus on elderly patients. One of the main aspects of this project is the system’s ability to properly interact with patients. This includes the ability to decode participants’ nonverbal cues of emotion and respond to such cues accordingly. The system will be equipped with a high resolution 2D/3D image capturing system to capture visual data and to decode such data using the appropriate algorithms. The adjustment of these algorithms will be carried out with the help of existing data bases and newly created material. This newly created material will consist of a range of videos filmed within similar therapist-patient scenarios. These will be analysed in order to investigate the frequency and quantity of nonverbal cues of emotion and the synchronisation of such cues (e.g. does the patient smile and when/how does the therapist react to this nonverbal cue?). Integration of these human interaction guidelines into the computer-based training system is expected to improve the efficiency of the training by creating an authentic nonverbal interaction between the training system and its patient/participant. Furthermore, the training system will be designed to be non-irritating in order to ensure the participant’s willingness to work with such a device. ‘Non-irritating’ in this case refers to the wireless nature of the aforementioned system and the possibility of working with the system in an environment without artificial lighting. Additionally, the system allows participants to train at home which could motivate them to express their true emotions due to an increased sense of privacy. This may be useful to the training system when capturing relevant information (facial expressions, gestures, body language).

The investigation of this aforementioned interaction between therapist and patient is the main aspect of the current sub-project called ‘Timing of Nonverbal and Emotional Expressions’. The successful nonverbal interaction between therapist and patient can be seen as an essential aspect in the area of health care. This aspect remains important even in the current time of digital revolution. With this revolution comes an increase in computer-based programmes assisting therapists in their work and offering home-based therapy for patients. The ability to decode as well as encode nonverbal cues of emotion may be beneficial for such digital training systems when interacting with patients. This sub-project seeks to investigate the exact timing of nonverbal cues in therapist-patient interactions by measuring the synchronisation of facial expressions and head movements in these dyadic interactions. Potential correlations between these nonverbal cues and age, gender or status (therapist/patient) of the participant/patient and his/her therapist are included in this investigation. All considered the aim of this sub-project is to gain an enhanced understanding of nonverbal aspects of therapist-patient interactions. This knowledge will be used to optimise the decoding and encoding abilities of the digital training system. Overall, the sub-project seeks to acquire a deeper understanding of human-computer interaction.

Contact:
Dr. phil. Uwe Altmann
Tel: +49 3641 936588