INSTITUTE FOR

INFORMATION TECHNOLOGY AND

COMMUNICATIONS

Das AI CoWorking Lab ist ein Verbund von 8 Forschenden: Prof. Dr. Ayoub Al-Hamadi (Neuro-Information Technology), Prof. Dr. Julia Arlinghaus (Production Systems and Automation), Prof. Dr. Benjamin Noack (Autonomous Multisensor Systems), Prof. Dr. Andreas Nürnberger (Data & Knowledge Engineering), SPRECHER Prof. Dr. Frank Ortmeier (Software Engineering), Prof. Dr. Myra Spiliopoulou (Knowledge Management & Discovery), Prof. Dr. Sebastian Stober (Artificial Intelligence) und Prof. Dr. Andreas Wendemuth (Cognitive Systems). Der Verbund ist eingebettet in die "Productive Teaming" Initiative (https://forschungsnetzwerk-chim.de/productive-teaming/) innerhalb des Forschungsnetzwerkes "Chemnitz-Ilmenau-Magdeburg (CHIM)" (https://forschungsnetzwerk-chim.de/).

Hauptziel des Gesamtantrages "AI Co-Working Lab" ist das Ermöglichen zukünftiger "Productive Teaming" Produktionssysteme, in denen Menschen und Maschinen auf Augenhöhe zusammenarbeiten. Das "AI Co-Working Lab" baut auf bestehenden Kompetenzschwerpunkten auf und nutzt Methoden der künstlichen Intelligenz.

Das vorliegende Forschungsthema dieses Teilprojektes "KI-gestützte intentionale Dynamiken in der Teamregulation" unterstützt das Gesamtziel direkt. Es erforscht eine zentrale Frage, nämlich die der Anforderungen an Mensch-Maschine-Interaktionen in einem hybriden Produktionsablauf. Human - Cyber Physical Production Systems (H-CPPS) stellen erhebliche kognitive Anforderungen an die menschlichen Partner, und für Effektivität und zielführende Zusammenarbeit bedarf es eines Interaktionsmodells, welches indirekt und/oder implizit kommunizierte Bedarfe erfasst und nutzt, was vor allem bei menschzentrierten und individualisierten Prozessen und Teamregulationen von zentraler Bedeutung ist. Dieses Modell wird hier erforscht und operationalisiert.

View project in the research portal

Im übergeordneten Projekt "Resiliente Human-Roboter-Kollaboration in Mixed-Skill-Umgebung (ENABLING)" (https://forschung-sachsen-anhalt.de/project/resiliente-human-roboter-kollaboration-27761) arbeiten die Arbeitsgruppen "Neuro-Informationstechnik" (Prof. Al-Hamadi) und "Kognitive Systeme" (Prof. Wendemuth). Das Vorhaben ENABLING adressiert den Problemraum der Entwicklung von KI-Methoden zur gegenseitigen Ergänzung der Skills von Roboter und Mensch.

Dieses Teilprojekt "Mensch-Roboter-Interaktion mit KI-Systemen" unter Leitung von Prof. Wendemuth hat die folgenden Forschungsfragen:

• Durchführung von Studien im Bereich (Sprach-)Interaktion im Bereich Mensch-Maschine/Roboter-Interaktion vor allem im Bereich Intent Recognition
• Konzeptionierung von Multimodalen, KI-gestützten Multi-User Interaktionsszenarien
• Entwicklung von Machine-Learning-Algorithmen zur Verbesserung der o.g. Szenarien
• Mitarbeit an der Entwicklung von Maßen zur Qualitätsabschätzung von natürlicher Roboter-Mensch-Interaktion
• Entwurf und Programmierung von Verfahren und KI-Methoden im Bereich der Sprachdialoge

View project in the research portal

Summary
“IMIQ – Intelligent Mobility Space in the District” is a project of the IMR – Intelligent Mobility Space Saxony Anhalt (https://niimo.ovgu.de/en/Intelligent+Mobility+Space.html), which will be based in the Science Harbor in Magdeburg. Over a period of 3 1/2 years (01/2024 - 12/2027, actual operational start 8/2024), the science harbor will become a future district in which new solutions will be developed in a needs-oriented manner, tested technically and informationally and implemented socio-economically . Key innovations include a digital work-life twin (DMLZ) and a real-world laboratory for intelligent mobility (RIM).

Ambitions
The aim is to develop and test innovative mobility and communication approaches. A digital work-life twin (DWLZ) enables a holistic and innovative mobility and communication experience that offers efficient and personalized solutions through sensors, 5G and digital services and at the same time promotes social interaction and exchange on site. In the Intelligent Mobility Real Laboratory (RIM), the researchers' developments on intelligent mobility become physically visible and tangible/experienceable; they are tested and evaluated. Technologies for communication and V2X, localization and tracking are controlled in an operation control center, integrated with infrastructure (including mobility stations) and with implemented autonomous vehicles.

Further Information
You can find a detailed description, news and staff positions here: https://niimo.ovgu.de/IMIQ.html. Under this link, or under the names linked above, you will also find information about the IMIQ work areas of the project partners.

This project will build up cutting-edge research in the interdisciplinary research field of mobility at the OVGU and enable the transfer of new mobility solutions in Saxony-Anhalt and beyond. The visibility or experience is aimed at all stakeholders.

View project in the research portal

Research content Mobile daily planning in the neighborhood:

Definition of data formats for a digital work-life twin
Development of AI-supported methods for mobility needs
Implementation of processes for work, services and mobility in the neighborhood
Science port for different target groups

Research content Individual public transport:

Collecting and making data available for individual mobility needs
Development of individualized dialog-based voice assistance systems for barrier-free interaction with a digital work-life twin
Exploring and experimenting with multimodal technologies
Application in the operation of autonomous electric shuttle buses

View project in the research portal

The Otto von Guericke University Magdeburg (research focus of Intelligent Mobility Space IMR, spokesman Prof. A. Wendemuth) and local transport service Sachsen-Anhalt GmbH (NASA GmbH) jointly create an experimentation space for mobility solutions in the Magdeburg area. Both sides signed a cooperation agreement in February 2021. New results and technologies from research will be tested and implemented for mobility & living/ living of the future. Everyday solutions are developed in a practical way to better network the city and the surrounding area. This is where individualized offers are created for elderly people who are limited in their mobility, as well as for young mobile families.

View project in the research portal

NIIMO: Netzwerkinitiative Intelligente Mobilität
Mobilitätsbedürfnisse, verkehrsplanerische und verkehrswirtschaftliche Ansätze, Reallabors, in Kooperation der OVGU mit der NASA GmbH. Dies wird mit Kooperationsvertrag OVGU-NASA vom Februar 2021 verfolgt.

Weiterführende Informationen
Seitens der OVGU wid NIIMO koordiniert vom Wissensverbund IMR - Intelligenter Mobilitätsraum Sachsen-Anhalt (https://niimo.ovgu.de/Intelligenter+Mobilit%C3%A4tsraum.html)

View project in the research portal

Kollaborationsfähige Robotersysteme sind eine Schlüsseltechnologie der flexiblen intelligenten Produktion, Logistik und Medizin, die sich im Sinne der Verknüpfung komplementärer Skills in einer eng verzahnten und potentialorientierten Zusammenarbeit mit dem Menschen, aber auch zur Substitution von Aufgaben und Fähigkeiten einsetzen lassen. Das Vorhaben ENABLING adressiert den Problemraum der Entwicklung von KI-Methoden zur gegenseitigen Ergänzung der Skills von Roboter und Mensch. Somit werden Innovationen in den Querschnittsbereichen Informationstechnologie und Key-Enabling-Technologie ermöglicht und die Grundlage für zukünftige Anwendungen in Mixed-Skill-Umgebungen in den Leitmärkten geschaffen. Das ENABLING wird die Kollaboration in Mixed-Skill-Arbeitswelten grundlegend verändern, indem Mensch und Roboter für das gegenseitige Verständnis von Prozessen, Handlungen und Absichten befähigt werden. ENABLING erhöht für die vollständige Informationsverarbeitungskette nicht nur die Effizienz in Produktion und Logistik, sie minimiert auch die Gefahren im Arbeitsprozess.

View project in the research portal

The overarching goals of ASAMI II are to understand the relationship between verbal dispositions and intentions to act, as well as the strategies of users of an assistance system in a multi-person situation. For this, the development, evaluation and optimization of the situation-related disposition recognition of the user through spoken language will remain in focus. This is expanded to include the component of recognizing action intentions in the dialogic environment. The evaluation of user characteristics is an important prerequisite for dialogue management. An informed disposition recognition is established, which is based on acoustic events, which can be derived from spectral, prosodic and paralinguistic features. The knowledge gained will flow directly into the recognition of action intentions and interaction style, which will be used for an adaptive, goal-oriented dialogue strategy. Complementary to this, the acoustic user signals are analyzed within a multi-person situation. For this purpose, the dynamic interplay between active and passive interaction parts (involvement) of a communication partner is analyzed on an acoustic level. Such dynamic changes are an integral feature of a conversation and provide information about the user's strategies. In particular, this interplay can be observed in a scenario consisting of an assistance system and several people. The focus here is on establishing adaptive problem-solving strategies.

View project in the research portal

The overarching goals of ASAMI II are to understand the relationship between verbal dispositions and intentions to act, as well as the strategies of users of an assistance system in a multi-person situation. For this, the development, evaluation and optimization of the situation-related disposition recognition of the user through spoken language will remain in focus. This is expanded to include the component of recognizing action intentions in the dialogic environment. The evaluation of user characteristics is an important prerequisite for dialogue management. An informed disposition recognition is established, which is based on acoustic events, which can be derived from spectral, prosodic and paralinguistic features. The knowledge gained will flow directly into the recognition of action intentions and interaction style, which will be used for an adaptive, goal-oriented dialogue strategy. Complementary to this, the acoustic user signals are analyzed within a multi-person situation. For this purpose, the dynamic interplay between active and passive interaction parts (involvement) of a communication partner is analyzed on an acoustic level. Such dynamic changes are an integral feature of a conversation and provide information about the user's strategies. In particular, this interplay can be observed in a scenario consisting of an assistance system and several people. The focus here is on establishing adaptive problem-solving strategies.

View project in the research portal

Adaptive strategies for assistance technologies in multi-party interactions (ASAMI) are support paradigms that can offer targeted technical assistance for individual or multiple actors in order to reduce uncertainty in action planning and in the joint interaction of the actors and to advance task processing. This includes anticipating and selecting options for action, monitoring and adapting the consequences of actions, strategies for obtaining information (external), situational exploration and communicative strategies such as feedback, informing, intervening or negotiating using means of multimodal, dialogic communication. Also included is the translation and creative linking of knowledge from other contexts in order to expand the scope of possibilities. Action-guiding objectives and plans of the actors are recorded and included.

View project in the research portal

Intentional, anticipatory, interactive systems (IAIS) represent a new class of user-centered assistance systems and are a nucleus for the development of information technology with corresponding SMEs in Saxony-Anhalt. IAIS uses action and system intentions derived from signal data, and the affective state of the user. By anticipating the further action of the user, solutions are interactively negotiated. The active roles of humans and systems change strategically, which requires neurological and behavioral models. The  human-machine-systems are being deployed in our systems lab, based on previous work in the SFB-TRR 62. The goal of lab tests is the understanding of the situated interaction. This supports the regional economy in their integration of assistance systems for Industry 4.0 in the context of demographic change.

View project in the research portal

ADAS&ME will develop adapted Advanced Driver Assistance Systems that incorporate driver/rider state, situational/environmental context, and adaptive interaction to automatically transfer control between vehicle and driver/rider and thus ensure safer and more efficient road usage. The work is based around 7 Use Cases, covering a large proportion of driving on European roads. Experimental research will be carried out on algorithms for driver state monitoring as well as on Human-Machine-Interaction and automation transitions. Robust detection/prediction algorithms will be developed for driver/rider state monitoring towards different driver states, such as fatigue, sleepiness, stress, inattention and impairing emotions, employing existing and novel sensing technologies, taking into account traffic and weather conditions and personalizing them to individual driver s physiology and driving behavior. Further, the core development includes multimodal and adaptive warning and intervention strategies based on current driver state and severity of scenarios. The final outcome is a driver/rider state monitoring system, integrated within vehicle automation. The system will be validated with a wide pool of drivers/riders under simulated and real road conditions and under different driver/rider states. This challenging task has been undertaken by a multidisciplinary European Consortium of 30 Partners, including an original equipment manufacturer per vehicle type and 7 direct suppliers.

The Cognitive Systems Group at Otto-von-Guericke-University will contribute to this consortium by providing analysis of emotional content of acoustic utterances in the car. We will also engage in information fusion of data from various modalities (acoustic, video, and others) and analyzing this data for identifying markers for detecting drowsiness or a loss of control state of the driver, thus contributing to driver assistance in several use cases, such as cars, busses, trucks, and motorcycles.

View project in the research portal

The alliance  3Dsensation gives machines  the ability of the visual recording and interpretation of complex scenarios through innovative 3D technologies. Machines are  situationally acting partners and personalized Wizard of Oz. The new form of human-machine interaction 3Dsensation manages access
to living and working environments regardless of age and physical fitness.

In the production 3Dsensation allows the symbiosis of man and machine on the basis of 3D vision. It creates a safe environment for people in manufacturing processes, ensuring the perception of assistance systems and ensures the quality of products.

Through the 3D collection and analysis of facial expressions, gestures and movement to control assistance systems 3Dsensation improve health care and guarantee self-determination into old age.

By coupling of 3D information with assistance systems 3Dsensation enables individual mobility independent of health and age-related impairments in urban and rural areas.

3Dsensation creates individual security by the autonomous experiential 3D analysis of characteristics of individuals and movements for the identification of problems and dangers.

By sector and interdisciplinary networking of science and industry an alliance is created, which solves key technical, ethical and sociological issues of human-machine interaction.

3Dsensation delivers fundamentally new solutions of human-machine interaction and thus secures the future for Germany's most important export industries.

View project in the research portal

The alliance 3Dsensation provides machines, through innovative 3D technologies, with the ability of visual recording and interpretation of complex scenarios.

Consequently machines become situational acting partners and personalized human assistants.
This new form of human-machine interaction enables access to living and working environments, regardless of age and physical capacity.

MOD-3D is going to investigate modeling of temporal sequences of behavior and action intentions
from multimodal 3D-Data and to analyze prospective improvements with regards to that.

View project in the research portal

The alliance 3Dsensation provides machines, through innovative 3D technologies, with the ability of visual recording and interpretation of complex scenarios.

Consequently machines become situational acting partners and personalized human assistants.
This new form of human-machine interaction  enables access to living and working environments, regardless of age and physical capacity.

MOVA3D realizes a novel multimodal omnidirectional 3D-Sensor for behavioural analyses of persons in ambient assisted living. Acoustic3D-Signals will be evaluated regarding situation und affectivity of the person.

View project in the research portal

The alliance 3Dsensation provides machines, through innovative 3D technologies, with the ability of visual recording and interpretation of complex scenarios.

Consequently machines become situational acting partners and personalized human assistants.
This new form of human-machine interaction  enables access to living and working environments, regardless of age and physical capacity.

MOD-3D is going to investigate modeling of temporal sequences of behavior and action intentions
from multimodal 3D-Data and to analyze prospective improvements with regards to that.

View project in the research portal

Fusion of situation data from speech, gesture, mimics, and psychobiological data   will give
advanced classification results.  Multimodal information fusion architectures are generated.

View project in the research portal

Emotions will be recognized from  speech. Features are subsymbolic and biologically inspired. Emotion classes are being identified.  Fusion with other information sources will give
advanced classification results. Intention recognition is a further goal. Emotional Speech will be provoked. Emotion-annotated data bases will be generated.

View project in the research portal

Fusionof situation data from speech, gesture, mimics, and psychobiological data   will give

advanced classification results.  Multimodal information fusion architectures are generated.

View project in the research portal

Emotions will be recognized from  speech. Features are subsymbolic and biologically inspired. Emotion classes are being identified.  Fusion with other information sources will give
advanced classification results. Intention recognition is a further goal. Emotional Speech will be provoked. Emotion-annotated data bases will be generated.

View project in the research portal

Prof. Wendemuth ist the Magdeburg Speaker of the SFB / TRR 62 "A Companion-Technology for Cognitive Technical Systems". Central Coordination is responsible for Project Management, two labs in Ulm and Magdeburg are co-ordinated,  3 demonstration scenarios will be built at both sites, Wizard-of-Oz-trials are being conducted. A Graduate Colleg will be installed.

View project in the research portal

The application-oriented research in the field " emotion -based support for interactive applications in call centers " will be further developed. Here is the phone dialogue ,
in which the call center operator is supported  in his conversation design through feedback on the
emotional state (control , valence ).

View project in the research portal

Category theory is used to identify dispositions and matching features. Appraisals are the main unit for analysis. As a result, a consolidated feature basis is obtained.

View project in the research portal

Hier soll ein Situationsmodell genutzt werden, um top-down Durchgriff im Spracherkenner und Dialogmanager zu ermöglichen. Ziel ist, nicht nur (dichte) Lattices als Schnittstellen zu nutzen, sondern z.B. bei Änderung der akustischen Umgebung direkt die akustische Merkmalsextraktion zu adaptieren und iterativ den Spracherkenner neu zu nutzen. Ähnliches gilt für Änderungen im Emotions- oder Verhaltenszustand, die z.B. zur Nutzung angepasster akustischer Modelle führen. Oder Änderungen in der Domäne oder der Aufgabe, oder der Kooperativität oder der Intention des Benutzers, die den Dialogmanager beeinflussen. Lernvorgänge sind hier zu implementieren und zu untersuchen bzw. die Anzahl von Alternativen zu vergrößern. Aus der Spracherkennung sind abgeleitete Grössen zu definieren, die für Verhaltensmodelle elevant sind und von diesem interpretativ verwendet werden können bzw. dieses modifizieren.

View project in the research portal

We used textual transcripts to analyse interaction styles and discourse structures. Further, we model the subject's internal success state with a hidden Markov model trained using the observed sequences of system feedback. Aiming on automatic detection of specic  subjects's reactions, we then semi-automatically annotate significant dialog events, i.e. phrases indicating an irregular, i.e. not-task-oriented subject behavior. We use both acoustic and linguistic features to build several trait-specic classiers for dialog phases.

View project in the research portal

A Model for localisation of  moods and personality traits in valence-pleasure-arousal-space is developed. Experimental trals are located in this space and a trajectory is modelled, which is mood- and personality dependent.

View project in the research portal

Zwei Modelle des assoziativen und kontextabhängigen Lernens werden modelliert. Damit können Versuche mit menschlichen Probanden, welche Teil der Arbeit von Prof. Dr. Jochen Braun und der Doktorarbeit von Dipl.-Ing. Oussama Hamid sind , informationstechnisch nachvollzogen werden. Die beiden Modelle verfolgen jeweils zwei unterschiedliche Ansätze und wurden in Matlab implementiert.

Ein Ansatz zur Modellierung basiert auf einem Markov-Entscheidungsprozess (engl. Markov Decision Process), wie er häufig im Bereich des Maschinellen Lernens verwendet wird. Ein damit entworfener menschenähnlicher Lernalgorithmus wurde anschließend um die Fähigkeit erweitert aus dem Zeitkontext in der Lernaufgabe Nutzen zu ziehen.

Der zweite Ansatz ist ein Kapazitätsmodell, welches sich auf Erkenntnisse aus der Gedächtnispsychologie stützt. Das Lernen von Assoziationen wird als Prozess im Kurzzeitgedächtnis modelliert, wobei der zeitliche Kontext unterstützend wirkt. Die Kapazität des Kurzzeitspeichers ist dabei der limitierende Faktor. Die Rolle der zeitlichen Information wurde auf verschiedene Weisen in das Modell implementiert. Es kann z.B. ein Einfluss auf die Vergessensrate oder auf das Erinnerungsvermögen der Probanden simuliert werden. Für die Simulation von Umlernen bei Kontextwechsel wurde zusätzlich ein Langzeitgedächtnis in das Modell eingefügt.
informationstechnisch nachvollzogen werden. Die beiden Modelle verfolgen jeweils zwei unterschiedliche Ansätze und wurden in Matlab implementiert.

View project in the research portal

Emotions will be recognized from  speech. Features are subsymbolic and biologically inspired. Emotion classes are being identified.  Fusion with other information sources will give

advanced classification results. Intention recognition is a further goal. Emotional Speech will be provoked. Emotion-annotated data bases will be generated.

View project in the research portal

Fusionof situation data from speech, gesture, mimics, and psychobiological data   will give

advanced classification results.  Multimodal information fusion architectures are generated.

View project in the research portal

Prof. Wendemuth ist the Magdeburg Speaker of the SFB / TRR 62 "A Companion-Technology for Cognitive Technical Systems". Central Coordination is responsible for Project Management, two labs in Ulm and Magdeburg are co-ordinated,  3 demonstration scenarios will be built at both sites, Wizard-of-Oz-trials are being conducted. A Graduate Colleg will be installed.

View project in the research portal

Zwei Modelle des assoziativen und kontextabhängigen Lernens werden modelliert. Damit können Versuche mit menschlichen Probanden, welche Teil der Arbeit von Prof. Dr. Jochen Braun und der Doktorarbeit von Dipl.-Ing. Oussama Hamid sind , informationstechnisch nachvollzogen werden. Die beiden Modelle verfolgen jeweils zwei unterschiedliche Ansätze und wurden in Matlab implementiert.

Ein Ansatz zur Modellierung basiert auf einem Markov-Entscheidungsprozess (engl. Markov Decision Process), wie er häufig im Bereich des Maschinellen Lernens verwendet wird. Ein damit entworfener menschenähnlicher Lernalgorithmus wurde anschließend um die Fähigkeit erweitert aus dem Zeitkontext in der Lernaufgabe Nutzen zu ziehen.

Der zweite Ansatz ist ein Kapazitätsmodell, welches sich auf Erkenntnisse aus der Gedächtnispsychologie stützt. Das Lernen von Assoziationen wird als Prozess im Kurzzeitgedächtnis modelliert, wobei der zeitliche Kontext unterstützend wirkt. Die Kapazität des Kurzzeitspeichers ist dabei der limitierende Faktor. Die Rolle der zeitlichen Information wurde auf verschiedene Weisen in das Modell implementiert. Es kann z.B. ein Einfluss auf die Vergessensrate oder auf das Erinnerungsvermögen der Probanden simuliert werden. Für die Simulation von Umlernen bei Kontextwechsel wurde zusätzlich ein Langzeitgedächtnis in das Modell eingefügt.
informationstechnisch nachvollzogen werden. Die beiden Modelle verfolgen jeweils zwei unterschiedliche Ansätze und wurden in Matlab implementiert.

View project in the research portal

Hier soll ein Situationsmodell genutzt werden, um top-down Durchgriff im Spracherkenner und Dialogmanager zu ermöglichen. Ziel ist, nicht nur (dichte) Lattices als Schnittstellen zu nutzen, sondern z.B. bei Änderung der akustischen Umgebung direkt die akustische Merkmalsextraktion zu adaptieren und iterativ den Spracherkenner neu zu nutzen. Ähnliches gilt für Änderungen im Emotions- oder Verhaltenszustand, die z.B. zur Nutzung angepasster akustischer Modelle führen. Oder Änderungen in der Domäne oder der Aufgabe, oder der Kooperativität oder der Intention des Benutzers, die den Dialogmanager beeinflussen. Lernvorgänge sind hier zu implementieren und zu untersuchen bzw. die Anzahl von Alternativen zu vergrößern. Aus der Spracherkennung sind abgeleitete Grössen zu definieren, die für Verhaltensmodelle elevant sind und von diesem interpretativ verwendet werden können bzw. dieses modifizieren.

View project in the research portal

Combining Modalities (with confidences) on the feature stream. Probabilistic Theory for thecorrect evaluation of the overall best hypothesis.

View project in the research portal

The project NIMITEK II adresses man-machine interaction. Neurobiological models will be generated by partners in Magdeburg, and will be used for modelling emotion and intention recognition which will in turn improve the user-friendliness of the machine system. Conversely, the technical system will be a test bed for neurological and behavioral research. Prof. Dr. Wendemuth is coordinator of the NIMITEK consortium. In his group, particular research aspects of the project are speech and prosody recognition, multimodal information extraction, classification of emotional units, modelling of associative coherence.

View project in the research portal

The overarching questions to be addressed by this project are as follows:

• Is the learning of context-conditional associations by human observers influenced by, or even predicated on, consistent temporal ordering of environmental events? In other words, can the context-dependence of human associative learning be understood in terms of a temporalorderdependence?
  
• How does temporal-order-dependent learning compare to abstract learning algorithms (e.g.,support-vector machines, dynamic adaptation of neural nets) for detecting patterns and regularities in high-dimensional data streams?
  
• Is temporal-order-dependent learning suited as a general solution to complex learning problems? How does it perform on diverse problems such as those described in section 7.3 (i.e., learning to recognize prosodic signals in speech or emotional markers in facial expression)?

View project in the research portal

The immediate goal is to analyze concurrent speech utterances and facial expressions in terms of speaker emotion and intention. Speech and face information will be combined to a multi-modal feature vector and subjected to blind source separation (ICA) analysis. In a different context similar methods were already suggested by the applicant in his Habilitationsschrift [Michaelis 80]. In the longer term, the proposed project is aimed at the automatic recognition of subtly different human interactions (e.g., friendly/cooperative, impatient/evasive, aversive/violent). A second long-term goal is to apply the automatic recognition of emotion states to a neurobiological investigation of the neural basis of emotion. A correlation with results of EEG and MRI investigations can be carried out [Heinzel 05]. The software tools to be developed here would be invaluable in brain imaging (fMRI) of human emotion.interactions (e.g., friendly/cooperative, impatient/evasive, aversive/violent). A second long-term goalis to apply the automatic recognition of emotion states to a neurobiological investigation of the neuralbasis of emotion. A correlation with results of EEG and MRI investigations can be carried out[Heinzel 05]. The software tools to be developed here would be invaluable in brain imaging (fMRI) ofhuman emotion.

View project in the research portal

Hier sollen das Situationsmodell und Ergebnisse des iterativen, einander modifizierenden top-down und bottom-up Prozesses in der Spracherkennung (Projekt Situationsangepasste Spracherkennung) genutzt werden, um ein interpretatives Verhaltensmodell einer Person oder von Personen in einer definierten Situation / Umgebung ( Situiertheit ) zu erzeugen und damit Interaktion als (intentionales) Verhalten zu modellieren. Die Ergebnisse des Projektes Situationsangepasste Spracherkennung dienen hier als direktes Maß dafür, wie sich die Person(en) zur Umgebung und zu einer gestellten Aufgabe äußern (Inhalt, Emotion) und wie dies mit den erfassten Umgebungsparametern zusammenpasst (match / mismatch der sprachlichen Äusserungen zur Umgebung), woraus Bestätigungen oder Änderungen des Verhaltensmodells abgeleitet werden können. Das gleiche gilt für eine Intentionserkennung, die mit B.Vlasenko zusammen entwickelt wird. Für die Situationsbeschreibung sind insbesondere Modellgrössen wie Zustandsparameter, Ziel(Kosten)grössen, Optimierungskriterien (LQ, ML, MMI, ME, MDL, andere?) zu definieren. Iterative und/oder syntaktisch-deskriptive (wenn-dann-Beziehungen) Lernvorgänge sind hier zu implementieren und zu untersuchen bzw. die Anzahl von Alternativen zu vergrößern. Das umfasst sowohl die Fähigkeit zum besseren Lernen einer Situation wie auch das Lernen, zwischen verschiedenen Situationen zu unterscheiden (dies ist auch in der Gruppe Prof. Braun von hohem Interesse). Aus dem Verhaltensmodell sind abgeleitete Grössen zu definieren, die für die Spracherkennung und Dialogmanager  relevant sind und von dieser interpretativ verwendet werden können bzw. diesen modifizieren.

View project in the research portal

This proposal is about using speech technology for the purpose of adding data to and requesting information from a remote database via telephone. Speech input to a data base is handled by automatic speech recognition technology while speech output is handled by speech synthesis technology. Inline with these two technologies, there are two major processes that are essential: speech understanding and response generation. Automatic speech recognition is not speech understanding, i.e., there is no linguistic or semantic analysis performed. In order to do live interaction with a remote database via telephone speech understanding is vitally important. Similarly speech synthesis cannot decide what to say by itself, therefore the response generation part should provide what is to be said by the speech synthesizer.

View project in the research portal

This proposal is about using speech technology for the purpose of adding data to and requesting information from a remote database via telephone.  Speech input to a data base is handled by automatic speech recognition technology while speech output is handled by speech synthesis technology. Inline with these two technologies, there are two major processes that are essential: speech understanding and response generation. Automatic speech recognition is not speech understanding, i.e., there is no linguistic or semantic analysis performed. In order to do live interaction with a remote database via telephone speech understanding is vitally important. Similarly speech synthesis cannot decide what to say by itself, therefore the response generation part should provide what is to be said by the speech synthesizer.

View project in the research portal

Support-Vector-Machines are used for flexible tailoringof automatic speech recognition systems to new tasks.

View project in the research portal

The project NIMITEK II adresses man-machine interaction. Neurobiological models will be generated by partners in Magdeburg, and will be used for modelling emotion and intention recognition which will in turn improve the user-friendliness of the machine system. Conversely, the technical system will be a test bed for neurological and behavioral research. Prof. Dr. Wendemuth is coordinator of the NIMITEK consortium. In his group, particular research aspects of the project are speech and prosody recognition, multimodal information extraction, classification of emotional units, modelling of associative coherence.

View project in the research portal

The project NIMITEK adresses man-machine interaction. Neurobiological models will be generated by partners in Magdeburg, and will be used for modelling emotion and intention recognition which will in turn improve the user-friendliness of the machine system. Conversely, the technical system will be a test bed for neurological and behavioral research. Prof. Dr. Wendemuth is coordinator of the NIMITEK consortium. In his group, particular research aspects of the project are speech and prosody recognition, multimodal information extraction, classification of emotional units, modelling of associative coherence.

View project in the research portal

Support-Vector-Machines and Kernel-based Methods will be utilized in automatic speech recognition. Numercial Methods will be used to generate probability measures.

View project in the research portal

Support Vector Machines are used to model Production probabilitieswhich are used as acoustic models in automatic speech recognition.

View project in the research portal

In this dissertation, various parametric estimation methods in automatic speech recognition are  researched.The aim is to develop estimation methods with high generalisation ability in speech recognition, in particular with few or mismatching data,as well as with disturbances by  noise, channel etc. The work is both theoretical and software-oriented.

View project in the research portal

Support-Vector-Machines and Kernel-based Methods will be utilized in automatic speech recognition. Numercial Methods will be used to generate probability measures.

View project in the research portal

In this dissertation, various parametric estimation methods in automatic speech recognition are  researched.The aim is to develop estimation methods with high generalisation ability in speech recognition, in particular with few or mismatching data,as well as with disturbances by  noise, channel etc. The work is both theoretical and software-oriented.

View project in the research portal

In robust speech recognition, phonetic units in signal space will be identified as belonging to one class. To ensure class separability, methods from digital signal processing wil be used. The new approach willtransform data implicitely into a high dimensionalfeature space, however only kernels will haveto be computed. Large matrix inversions will have to be done iteratively.

View project in the research portal

In robust speech recognition, phonetic units in signal space will be identified as belonging to one class. To ensure class separability, methods from digital signal processing wil be used. The new approach willtransform data implicitely into a high dimensionalfeature space, however only kernels will haveto be computed. Large matrix inversions will have to be done iteratively.

View project in the research portal

The summer school will:- provide concise methodological knowledge on speech recognition techniques to young researchers who have not yet had in-depth experience in the field- provide in-depth knowledge of robust methods - have young researchers apply their attained knowledge immediately in existing speech recognition software systems- enable young researchers to design their individual speech recognition application, in their own language on existing corpora- open  up contacts to other academic institutions and industry through invited lecturers and application-oriented collaborations on site

View project in the research portal

Participants will take part in lecture and lab courses in Digital Signal Processing  and Speech Recognition. They will then apply their knowledge within the speech recognition computer lab installed at the groups premises. Here, Texas Instruments  Digital Signal Processors, Matlab tools for digital signal processing, and the Hidden Markov Toolkit (HTK) for speech recognition is available and running. Students will work in close collaboration with group members on state-of-the-art problems in digital signal processing and  speech recognition. Results of their work will be integrated into the groups architecture.

View project in the research portal