Virtual companion "Josy" cares about elderly people

“She” is ticklish, loves to laugh, and drinks regularly: “Josy” is a virtual assistance technology system designed to help older people in their everyday lives. The system was developed by Franziska Braun and her colleagues at the University of Stuttgart's Institute of Human Factors and Technology Management (IAT) and the Fraunhofer Institute for Industrial Engineering (IAO).

“Josy acts as a friend that older people like to talk to and include in their daily lives,” Braun explains. For example, the virtual companion regularly takes a drink, which is something that older people in particular often forget to do. The system can also recognize when someone feels lonely or sad, whereupon it asks: “Would you like to call your relatives, or should we send them a picture of you?” The technology is intended as an aid in preventing loneliness and social isolation.

Josy incorporates an app based on facial recognition technology, whereby the built-in camera scans a face and compares the resulting data with an emotion recognition library. If the corners of the mouth point downwards then the system assumes that person is sad and responds by suggesting something along the lines of looking at a virtual photograph album together. Relatives can select the images beforehand and store them in the app. The fact that Josy can be personalized is important to Braun and her team: “Our aim is to create a personalized solution that can create an emotional connection. Everyone likes it when something comes from the heart.” That's why the researchers also offer sewing instructions for making a cover for Josy, which relatives can use to customize the companion even more.

The project is part of the High Performance Center for Mass Personalization (LZMP), where research is focused on a user-centric approach: “We've been engaging a lot with our potential users, always listening to what they're thinking and trying to understand their thought processes.” In the process, we discovered that older people are often worried that they might make mistakes or break something when using digital devices. “That's why we programmed Josy to be as simple as possible to use,” says Braun: “No one will be able to accidentally delete the Internet or buy anything by mistake.”

A code has to be entered in order to access the device settings area, which relatives and friends can use to add photos or change the viewing settings, for example. Among other things, Braun and her team acquired these insights in a series of online workshops, the aim of which is to present Josy as an assistance solution to interested persons, but a lot of it is also about dialog.

The workshops are aimed at a range of very different target groups including managers who are interested in innovative production processes, care facilities that want to learn more about the system, companies that are interested in the product, and, of course, people with elderly relatives.

Braun makes a point of ensuring that the training seminars do not resemble a traditional classroom teaching session and is pursuing a community-based approach with the project: “Engaging in dialog with interested parties and the public is hugely important to us,” she explains. “We explicitly invite public participation and plan to incorporate the feedback into our research, which will ultimately benefit us and the project.”

Scientists from nine University of Stuttgart Institutes and four Fraunhofer Institutes in Stuttgart are collaborating with future-oriented industrial partners to explore ways of manufacturing customized products for the mass market. The concept of “Mass Personalization” takes account of the individual needs of users right from the product development stage.

Editor: Bettina Wind