These are the associated projects that are related to the research questions studied within our ONE MUNICH Forum Project “Next Generation Human-Centered Robotics”:
- AI.D – Artificial Intelligence for Neuro Deficits
- eXprt – Exoskeleton and Wearables Enhanced Prevention and Treatment
- Interactive Learning of Explainable, Situation-Adapted Decision Models
- Learning-Aided Real-World Semantic Exploration
- Limitations of Deep Neural Network
- Remaining Useful Lifetime for New and Used Technical Systems under Non-Stationary Conditions
- The Curious Robot: An Unsupervised Human-in-the-Loop Action-Learning Approach
- TransforM – Transparenz für Maschinen in Persönlichen intelligenten Umgebungen
- 6G-life Digital Transformation and Sovereignty of Future Communication Network
Previous Projects:
AI.D – Artificial Intelligence for Neuro Deficits
Limb loss, spinal cord injury (SCI), stroke, neuromusculoskeletal disorders (NMD), multiple sclerosis (MS), and cerebral palsy (CP), whose combined prevalence is approximately 3.8%, are all conditions that affect individuals ability to use their limbs without assistance. Prostheses, exoskeletons, and robotic assistive systems are promising means for chronic patients to regain their autonomy. In this context, there is a strong need for designing systems that recognize human intent and mimic the intended behavior as close as possible to the natural limb, in order to maximize the therapeutic effect and patient benefit.
Associated ONE MUNICH PIs Prof. Dr.
Simon Jacob(TUM)
Prof. Dr.
Sami Haddadin(TUM)
Simon Jacob
Sami Haddadin
The project AI.D aims to restore lost function of neurologically and neuromuscularly impaired patients through a new generation of human-model-informed and learning-enabled neuro machines, designed and controlled according to the principles of human neuromechanics and motor control, using a Brain/Body-Machine Interface (BMI). The mission of this translational research project is to develop novel methods and technologies that substantially improve the state of the art and provide support to such patients in the following three central phases:
- Receiving assistance and mobilization care
- Regaining mobility and independence
- Reinclusion into society
In order to achieve this ambitious goal the strategic approach will be based on three pillars:
- Brain-Machine Interface: Human model-informed causal signal processing
- Human-informed AI: Intelligent control and learning algorithms
- Clinical studies: From technology to clinical validation
Based on our pioneering work in robotics, neural control, neuromechanics modeling and physics-informed machine learning, the innovative special feature of AI.D project is the development of a new generation of AI-enhanced human-model-informed neuro machines, designed and controlled according to the principles of the human body, to restore autonomy and mobility to the physically challenged – amputees, paralyzed, and stroke patients.
eXprt – Exoskeleton and Wearables Enhanced Prevention and Treatment
The Innovation Network eXprt brings together a multidisciplinary team to create a synergy of engineering, neuroscience, and clinical neurology to develop wearable technologies. The project provides tools to sensitively identify sensorimotor and cognitive impairments in everyday life. Through the power of neuroengineering, the results of eXprt will efficiently compensate for lost motor function and, therefore, prevent deterioration and improve the quality of life of individuals in need.
Associated ONE MUNICH PIs
Sandra Hirche
eXprt will develop a robotic exoskeleton to assist individuals with impaired hand function. Furthermore, using neuroimaging methods, eXprt will examine the neuroplastic adaptations that come with the use of the exoskeleton. Additionally, via wearable technologies, eXprt will use state-of-the-art machine learning methods to analyse everyday behavior in larger cohorts to learn about how to best improve prevention, outcome, and secure independency in daily life.
For more information, please visit the website.
Interactive Learning of Explainable, Situation-Adapted Decision Models
The focus of this project investigates a novel approach through which the space of possible models explaining a certain decision can be explored interactively by a user until a model is found that satisfies the needs of the user in terms of the trade-off between accuracy and model complexity. The project defines and explores a refinement relation that defines a lattice as an explanation space from which explanations can be selected.
For more details, visit the project webpage.
Associated ONE MUNICH PIs Prof. Dr.
Eyke Hüllermeier(LMU)
Eyke Hüllermeier
Learning-Aided Real-World Semantic Exploration
The goal of this 6-month project will be to combine our expertise in visual-inertial SLAM (VI-SLAM), dense and semantic mapping, as well as safe navigation and control, to achieve real-world exploration with a drone at an unprecedented scale, robustness, and speed. To achieve this, we will be tightly integrating probabilistic VI-SLAM with mapping and control, and furthermore leverage learning-based completion and semantic understanding for efficient and safe exploration. The drone platform as well as some core algorithms have been developed by the teams, which makes this project feasible in the short timeframe of 6 months.
Associated ONE MUNICH PIs
Stefan Leutenegger
This project is funded by MIRMI Seed Funds.
Limitations of Deep Neural Networks
The success of Deep Learning in many different practical application fields, ranging from image classification, protein folding prediction to natural language processing, lead to the (ongoing) development of a rich mathematical theory. Although great strides have been made to deepen our understanding of the field, many questions remain open at the moment. One of the most important but also most fundamental issues concerns the capabilities and limitations of Deep Learning. Simply put, which problems can we reasonably expect Deep Learning to solve and where can we, with great certainty, predict failures of Deep Learning methods. An often neglected aspect of this discussion are the restrictions imposed by the hardware the systems are running on. Deep Learning methods can not exceed the fundamental barriers of its computation platforms. Because of this it is crucial to link the capabilities of Deep Learning to (actual or theoretical) computation devices. The aim of this project is to characterize the possibilities and boundaries of Deep Learning inflicted by computational limits.
Associated ONE MUNICH PIs Prof. Dr.
Gitta Kutyniok(LMU)Prof. Dr.
Holger Boche(TUM)
Gitta Kutyniok
Holger Boche
Remaining Useful Lifetime for New and Used Technical Systems under Non-Stationary Conditions
Condition-based maintenance and predictive maintenance are increasingly applied in the industry due to their ability of ensuring an optimum utilization of the monitored system. These maintenance strategies allow for diagnosing and predicting the health states of the system under stationary operating conditions. However, technical systems mostly operate under non-stationary conditions, e.g. a wind turbine affected by different loads and speeds due to stochastic wind excitation. Non-stationary conditions lead to changed sensor data and thereby mask alterations caused by either faults or degradation of the system. Therefore, condition monitoring methods need to be extended and adapted for systems operating under non-stationary conditions.
Associated ONE MUNICH PIs Prof. Dr.
Eyke Hüllermeier(LMU)
Eyke Hüllermeier
The proposed project aims to develop methods for remaining useful lifetime prediction for systems operated under non-stationary conditions. Therefore, classical data-driven and model-based approaches from engineering are combined with approaches from the field of artificial intelligence. By a hybrid combination of clustering and classification with knowledge-based approaches, operating conditions are categorized and failure modes are identified. Based on uncertainty quantification and analyzed relationships between the operating conditions, the sensor data und the degradation evolution, suitable features for enabling the prediction of the remaining useful lifetime are developed and evaluated. Embedding non-stationary future operating conditions is realized by the use of different machine learning methods such as learning on data streams. These methods enable incremental learning and adaption to changes like variation of operating conditions. Moreover, hybrid methods are developed to allow a prediction of the remaining useful lifetime for used systems that are retrofitted with suitable sensors but lack sensor data of their past operation.For validation of the methods for remaining useful lifetime predictions, three application examples are chosen which have been selected from various thematic fields. To generate data for the first example, a suitable ball bearing test rig needs to be developed and constructed. The test rig should allow varying operating conditions regarding speed and bearing load. Run-to-failure data is acquired by different sensors such as acceleration sensors, temperature sensors, and strain gauges. For the second example, a laboratory experiment based on piezoelectric transducers is also implemented, whose failure is characterized by cracks and should be monitored. The third example is based on simulated data of a turbofan engine whose degradation under six conditions has been detected by various sensors.
The Curious Robot: An Unsupervised Human-in-the-Loop Action-Learning Approach
Domestic robots can bring the next step in human-computer collaboration, envisioned to allow many shared tasks such as cooking and cleaning. However, understanding the many ways humans perform actions is an unsolved problem. We propose a curiosity-driven robot that will learn user behavior based on videos. We use human pose estimation in combination with dimensionality reduction to understand the pose space. Using unsupervised clustering, we will find new unknown actions. As soon as a new action cluster emerges, the robot will ask for a label for this cluster and, thus, extend the knowledge graph with this human-in-the-loop approach.
Associated ONE MUNICH PIs Prof. Dr.
Albrecht Schmidt(LMU)
Albrecht Schmidt
TransforM – Transparenz für Maschinen in Persönlichen intelligenten Umgebungen
Summary
The TransforM project is a follow-up project to the SPP 2199 project PerforM, in which personality was investigated as an interaction paradigm for robotic intelligent spaces. PerforM has developed a so-called “room intelligence”, i.e. an intelligent living environment with robotic elements and intelligent kitchen appliances that can be addressed as a coherent unit and has a certain personality towards its users. Research into this paradigm in user studies revealed a conflict between the desire for transparency in such an environment (the user can understand what is going on behind the scenes) and the desire for technology to be invisible, especially in the home (the user wants things to simply work and not be concerned with all the details, a cosy atmosphere).
Associated ONE MUNICH PIs
Andreas Butz
The right degree of transparency is therefore crucial for the acceptance of such an environment. Building on transparency concepts from the field of Explainable AI and interpersonal communication, we want to investigate how transparency can be constructed and communicated in personal intelligent environments such as spatial intelligence and beyond. To this end, we want to design a language and toolbox of transparency building blocks and evaluate them in several user studies. Among other things, these studies will provide insights into how the transparency building blocks are perceived by users, what relationships exist between transparency and trust and acceptance, and whether it is possible to identify a “transparency sweet spot”. Furthermore, we will investigate the potential of “Adaptable Transparency”, i.e. adapting the level of transparency to individual characteristics and preferences of users (e.g. technical expertise, personality differences). Based on these studies, we will extract general scalable interaction paradigms and design guidelines for future smart spaces that ensure that they are neither too opaque nor too complex and – through the right level of transparency – generate an appropriate, calibrated level of trust.
This is a follow-up project of the previous project “PerforM – Personalities for Machinery in Personal Pervasive Smart Spaces“. For more information, please visit the website.
6G-life Digital Transformation and Sovereignty of Future Communication Network
Summary
TUM and the Technical University of Dresden have joined forces to form the 6G-life research hub to drive cutting-edge research for future 6G communication networks with a focus on human-machine collaboration. The merher of the teo universities of excellence combines their world leading preliminary work in the field of Tactile Internet in the Cluster of Excellence CeTI, 5G communication networks, quantum communication, Post-Shannon theory, artificial intelligence methods, and adaptive and flexible hardware and software platforms.
Associated ONE MUNICH PIs Prof. Dr.
Holger Boche(TUM)Prof. Dr.
Sandra Hirche(TUM)
Holger Boche
Sandra Hirche
Vision
6G-life will significantly stimulate industry and the startup landscape in Germany through positive showcase projects and thus sustainably strengthen digital sovereignty in Germany. Test fields for two use cases will drive research and economic stimulation. The goal is to create at least 10 new startups through 6G-life in the first four years and to involve at least 30 startups. 6G-life will significantly contribute to the creation of a skilled workforce. In addition, 6G-life has set itself the task of accompanying the population in the digital transformation and thus making a contribution to society.
For more details, visit the project webpage.
Previous Projects
PerforM – Personalities for Machinery in Personal Pervasive Smart Spaces
Condition-based maintenance and predictive maintenance are increasingly applied in the industry due to their ability of ensuring an optimum utilization of the monitored system. These maintenance strategies allow for diagnosing and predicting the health states of the system under stationary operating conditions. However, technical systems mostly operate under non-stationary conditions, e.g. a wind turbine affected by different loads and speeds due to stochastic wind excitation. Non-stationary conditions lead to changed sensor data and thereby mask alterations caused by either faults or degradation of the system. Therefore, condition monitoring methods need to be extended and adapted for systems operating under non-stationary conditions.
Associated ONE MUNICH PIs Prof. Dr.
Andreas Butz(LMU)
Andreas Butz
The proposed project aims to develop methods for remaining useful lifetime prediction for systems operated under non-stationary conditions. Therefore, classical data-driven and model-based approaches from engineering are combined with approaches from the field of artificial intelligence. By a hybrid combination of clustering and classification with knowledge-based approaches, operating conditions are categorized and failure modes are identified. Based on uncertainty quantification and analyzed relationships between the operating conditions, the sensor data und the degradation evolution, suitable features for enabling the prediction of the remaining useful lifetime are developed and evaluated. Embedding non-stationary future operating conditions is realized by the use of different machine learning methods such as learning on data streams. These methods enable incremental learning and adaption to changes like variation of operating conditions. Moreover, hybrid methods are developed to allow a prediction of the remaining useful lifetime for used systems that are retrofitted with suitable sensors but lack sensor data of their past operation.For validation of the methods for remaining useful lifetime predictions, three application examples are chosen which have been selected from various thematic fields. To generate data for the first example, a suitable ball bearing test rig needs to be developed and constructed. The test rig should allow varying operating conditions regarding speed and bearing load. Run-to-failure data is acquired by different sensors such as acceleration sensors, temperature sensors, and strain gauges. For the second example, a laboratory experiment based on piezoelectric transducers is also implemented, whose failure is characterized by cracks and should be monitored. The third example is based on simulated data of a turbofan engine whose degradation under six conditions has been detected by various sensors.
For more details, visit the project webpage.