Researchers at the University of Bonn have trained an algorithm to predict the course of a cooking recipe without having prior knowledge of it. Although still very imperfect, the program has been shown to be able to anticipate actions over several minutes.
For decades we have dreamed of a robot butler who knows what we need and when. For now, we have to be content with digital sticky notes that remind us of an appointment or offer us automatic responses to our emails.
But researchers at the University of Bonn in Germany say they have taken the first step towards creating a self-learning program capable of anticipating our actions, for the time being in the kitchen. Two neural networks were fed with four hours of videos showing humans preparing different salads. These data were used to train an algorithm on the twenty actions required for the preparation of each salad, the beginning and the duration of the recipe.
A prediction that is correct 4 times out of 10
The researchers then tested their AI’s learning process by submitting videos she had never seen. To help him, a single salad-making video was described for about 30% of its duration. The program then had to predict what would happen in the rest of the clip. Result? The algorithm was able to predict the following actions with an accuracy of 40%. This is obviously not an exceptional performance, but it is a notable first step forward.
To strengthen the voice assistants and robots
“Our methods are the first to predict the content of a video over a period of several minutes”, we can read in the scientific article which describes this project which will be presented at the Computer Vision and Pattern Recognition conference in Salt Lake. City (United States) next week. The accuracy of the predictive system decreases the further the action is from the starting point, but it still stays at 15% for activities more than three minutes into the future. Besides salads, AI has also been trained with videos of people preparing other dishes with similar success.
Once this technology is successful, it could help improve interactions with virtual assistants and robots. “We want to be able to predict when and how long activities will take minutes and even hours before they happen,” says one of the researchers. If that ever becomes a reality, we could live in smart homes that can prepare the ingredients and preheat the oven before we arrive hungry after a day’s work.