AI composes personalized sound to help you fall asleep

Lullabies exist in almost all cultures - and they are not only effective for children. But not every lullaby is equally effective for everyone. Personal preferences certainly play a role.

"My nephew was a Queen fan at the age of three, much to his parents' surprise," says Professor Dirk Pflüger from the Institute for Parallel and Distributed Systems at the University of Stuttgart. "He didn't want to listen to anything else at bedtime and fell asleep quickly. That wouldn't work for me."

However, the specific characteristics that music or acoustic stimuli must possess to help individuals sleep better have not yet been thoroughly researched. "We are trying to find out whether these are noises, whether you need particular rhythms or something similar," says Stuttgart-based computer scientist Pflüger.

With "we", he means an interdisciplinary team of researchers from the fields of musicology, neuroscience, psychology and computer science from nine European universities. Together they examine acoustic stimuli with different frequencies, from white to pink to brown noise. The latter sounds like rustling leaves or heavy rain. Or even artificially generated melodies and rhythms, such as those used in a sleep study as part of Lullabyte.

"The main aim is to teach an AI system to optimize acoustic stimuli based on my reaction to what I hear, so that they help me fall asleep in the best possible way," explains Pflüger. This requires machine learning methods that teach AI exactly that. Pflüger, a computer scientist from Stuttgart, is working on this together with his doctoral students Samuel Morgan and Jonathan Stumber.

These methods must be fed with input. "Ultimately, we need a broad data set," explains Pflüger. The data currently available comes from sleep laboratories and is very sparse. For Pflüger and his colleagues, one thing is certain: They must venture into people's home environment.

With mobile EEG devices, known as headbands, researchers would be able to measure brain activity and vital parameters, such as pulse and muscle activity, practically anywhere while asleep and share them easily for research purposes. "We are currently working on selecting and testing suitable devices for this," says Pflüger.

Another challenge is determining what the test subjects and the AI should ultimately listen to. At Aarhus University, researchers analyzed numerous Spotify playlists associated with sleep to identify correlations. The result: A repertoire of around 130,000 songs, melodies and sounds that are now being analyzed for their musical characteristics. "We use this data to feed an AI system that can learn to generate acoustic stimuli," explains Pflüger. "We play these generated stimuli to test subjects, measure their reactions, and feed the results back to the AI system. This iterative process continues until the acoustic stimuli are optimized to facilitate the best possible sleep."

The training of early career researchers plays an important role in the EU's Lullabyte research program. Pflüger and his colleagues applied for the project as a Marie Skłodowska-Curie network program (MSCA) for doctoral researchers and quickly received approval. Each of the participating universities provides a professor-doctoral researcher duo.

The doctoral researchers are sent to a partner university for two research stays, each lasting three months. "This strengthens international mobility and exchange", says Pflüger. The project also offers weekly doctoral researcher meetings and three summer schools.

As a Marie Skłodowska-Curie Actions (MSCA) doctoral network, Lullabyte will receive funding from Horizon Europe for four years, amounting to total funding of almost 2.5 million euros. Supporting young doctoral researchers is an essential part of the project. TU Dresden and FU Berlin lead the project, with involvement from eight other partner universities: Radboud University Medical Center in the Netherlands, University of Stuttgart in Germany, Aarhus University in Denmark, FEMTO-ST Institute and Paris Brain Institute in France, Universidad Pompeu Fabra in Spain, Royal Institute of Technology in Sweden, Université de Fribourg in Switzerland, and the Berlin-based start-up Endel.