A new level of human-machine interaction

Andreas Bulling is researching intelligent assistance systems at the University of Stuttgart, which are designed to provide people with the best possible support in areas such as medical diagnostics, care or as “everyday helpers”. In the future, they should even be capable of understanding and empathizing with others. The Alexander von Humboldt Foundation has named Bulling a Henriette Herz Scout for his expertise in the field of human-machine interaction and his commitment to promoting early career researchers.

In the future, intelligent assistance systems will work more and more closely with humans - whether in care, medical diagnostics or everyday life. They are already able to recognize body language, facial expressions and eye movements and can tell whether we are sad or happy, for example. However, they are not yet capable of fully understanding another person’s perspective or responding empathetically.

“To do this, we have to teach these systems to recognize our needs, goals and intentions,” says Prof. Andreas Bulling, an expert in human-machine interaction and cognitive systems at the University of Stuttgart. This means that they need significantly better sensory and cognitive skills. “The vision is to create intelligent assistants or avatars that we can work with like humans,” says the scientist.

In 2018, an ERC Starting Grant provided Bulling with the opportunity to begin researching his vision. Since then, the computer scientist has supported 14 doctoral students and postdocs in their academic careers. Seven of them are now working as professors themselves, and he is currently supervising 16 other doctoral students in his research group. Bulling has now been appointed a Henriette Herz Scout [de] by the Alexander von Humboldt Foundation - the second Stuttgart scout after Professor Johannes Kästner. 

In his role as scout, Bulling can nominate three young international researchers for a two-year research fellowship in his Collaborative Artificial Intelligence (CAI) department at the Institute for Visualization and Interactive Systems (VIS). “This is a unique sponsorship to attract young talent to the University of Stuttgart,” says Bulling, who himself spent two and a half years as a post-doctoral researcher at the University of Cambridge in England thanks to a Feodor Lynen Fellowship from the Humboldt Foundation. 

Funding opportunities at post-doctoral level are rare and involve a lot of bureaucracy and time. Bulling says that they often lose out to other universities in the race for top-qualified young talent. “With the Humboldt Fellowship, it is possible to start research with us within just a few weeks of being nominated.” 

He already has promising candidates in mind, says Bulling, though he prefers not to disclose any details just yet. He would like to integrate the new Humboldt Fellows in the field of cognitive modeling, among others. “We are currently conducting extensive research to enable intelligent agents to develop a theory of mind,” explains Bulling.

Theory of Mind refers to the cognitive ability to empathize with others. In concrete terms, this means understanding mental states as a possible cause of observed behavior and then being able to explain and predict one's own social actions or those of the other person. To do this, it is necessary to correctly recognize and understand the thoughts, feelings, intentions and expectations of the behavior of others. Most people use this skill naturally in interactions with others. However, teaching computers similar skills is extremely complex.

If Bulling and his team are successful with this, it would be possible for the first time to develop assistance systems that can perform actions proactively for the first time. “In the future, such an assistant could, for example, recognize that a patient needs help with an everyday task and proactively offer support.” Assistance systems would thus be able to respond to the emotional state and needs of their human counterpart. 

In Stuttgart, Humboldt Fellows benefit from a comprehensive computer science department situated at the crossroads of two Clusters of Excellence - SimTech and IntCDC - and the ELLIS Unit, which Bulling co-leads as one of its founding directors. “We are among the few interdisciplinary research groups in Germany and Europe specializing in human-machine interaction and cognitive modeling,” emphasizes Bulling.

In addition to research, the fellows will also be able to gain experience in teaching and supervising doctoral students as well as acquiring external funding. “The development of young scientists is very important to me, which is why I provide comprehensive and engaged support,” says Bulling.

The Henriette Herz Scouting Program is designed for researchers at all career stages and from a wide range of disciplines across Germany. To qualify as a scout, candidates must hold a (junior) professorship or a comparable leadership position, such as a group leader, and possess an established international collaboration network. Scouts identify excellent young researchers from abroad and can nominate up to three international scientific talents for a two-year research fellowship. Following a formal evaluation, up to 100 scholarships are awarded annually through a direct selection process.

2020 the University of Stuttgart has been presented with the Henriette Herz Award by the Alexander von Humboldt Foundation - an award for strategically oriented recruitment and forming closer bonds with highly qualified international early career researchers.