AI-powered nutrition for healthy aging
Researchers at the National Research Council of Canada (NRC) and University of Waterloo (UWaterloo) recently unveiled a new AI-based tool that makes calorie counting as easy as pie.
Calculating calories is about more than just losing or gaining weight. For example, tracking eating patterns helps detect possible dietary changes to improve nutrition and extend lifespan. This is particularly important for older adults looking to prevent or manage dementia, continue to live well and age in place.
While methods exist for monitoring, measuring and reporting nutritional intake based on stills or video, they can be less reliable or inaccurate. Some are a lot of work for the user. This new AI tool was designed to provide greater accuracy, simplify the monitoring process and lighten the load of self-reporting.
"Innovations that can assess food intake and diet quality in an accessible and inclusive way could become game-changers in our approach to healthy eating and healthy aging, addressing top risk factors for dementia and frailty," says Patricia Debergue, the NRC's director of the Aging in Place Challenge program. "This new tool could help empower older adults to continue living well and independently in their homes and communities as well as prevent transitions in care."
And the NRC–UWaterloo team's new tool has the means to serve up the right recipes.
Counting and classifying calories
While AI models for measuring calories and nutritional value have existed for some time, the newly developed NRC–UWaterloo technology outpaces anything the world has seen.
"Two versions of our model were highlighted at the 2024 MetaFood workshop, the first food workshop ever held at the prestigious Conference on Computer Vision and Pattern Recognition (CVPR)," says Dr. Pengcheng Xi, a senior research scientist at the NRC's Digital Technologies Research Centre and adjunct assistant professor at UWaterloo.
The team's model for predicting calories, mass, protein, fat and carbohydrates from images of a meal showed a 25.5% improvement over the current state-of-the-art model, according to the researchers' May 2024 article on nutrition prediction using food images (PDF, 1.04 MB). This will lead to much greater accuracy in dietary estimation.
The AI model even includes a new twist—segmenting and measuring food portions on utensils such as spoons, forks and chopsticks. Instead of examining an image of food on your plate, this model studies every bite on its way to your mouth and has only a 4.4% margin for error, according to an article on food portion estimation on spoons published by the researchers in May 2024 (PDF, 1.8 MB). Eventually, researchers will teach this AI technology to identify the different types of food on your utensils.
The NRC–UWaterloo team also organized an exciting and well-attended physically informed 3D food reconstruction challenge at the 2024 CVPR conference. International teams competed on developing different AI solutions that were tested on the data provided. According to Dr. Xi, the NRC will be able to use some of those models in the project. And the challenge winner has open-sourced their model code.
"The collaboration between our Vision and Image Processing research group and Dr. Xi's team at the NRC, supported by the Aging in Place Challenge program, has led to impactful AI innovations with global potential in health and wellness for the aging population," says Dr. Alexander Wong, Professor at UWaterloo. "This partnership is already accelerating international initiatives, and we look forward to continuing our research efforts with the NRC on more exciting innovations."
At the 2023 CPVR conference, the team published a paper about a third model that separates different types of food on a plate. This activity is critical to estimating total nutritional value but is immensely challenging because food on a plate inevitably gets mixed together. According to the 2023 article by the NRC and UWaterloo researchers comparing the new model with the previous one, their new model is 4.3% more accurate than the previous one.
"This collaboration has led to ground-breaking technological advancements that are recognized internationally, and has also showcased the prominent role of Canadian researchers in the global AI for food research community," shares Dr. Yuhao Chen, Research Assistant Professor at UWaterloo.
Stretching benefits into the future
The possibilities for using the AI model are endless. In addition to helping individuals manage their own nutrition, the model provides data that can be used by nutritionists and dieticians to create more personalized and effective dietary plans. For caregivers of older adults, it can provide clear insights into their dietary habits and lead to better management of their nutritional needs.
At the moment, the system can't distinguish foods or estimate their nutritional content, but the team will be integrating large language models to recognize a wide range of food. "This means people can ask the model questions about their progress—maybe even get advice," adds Dr. Xi. He also expects that, eventually, people will be able to use wearable technology such as smart glasses to video record their eating activities.
"This project has demonstrated its potential to enhance the quality of life for older adults and support the broader Canadian health care system in managing and improving nutritional health," says Debergue. "By addressing this important modifiable risk factor, it is contributing to the Aging in Place Challenge program vision."
To this end, the NRC continues to develop innovative technologies and solutions that support safe, healthy and socially connected living for older adults and their caregivers and offer collaboration opportunities for like-minded partners in the private, public and academic sectors.
This project is supported by grants and contributions awarded through the Collaborative Science, Technology and Innovation Program, administered by the NRC's National Program Office.