Google PhotoScan AI Camera Tool Estimates Body Fat Better Than Galaxy Watch Sensors

Google built a camera-based tool to estimate body fat more accurately than wearables

Beyond just breaking down body composition, the system could also be used to predict insulin resistance. Most of us have a weight problem. According to the NIH, almost 3 out of 4 American adults (73.1%) are either overweight or obese. But getting the full picture of your body composition is a whole lot more complicated than just stepping on a scale, and some of the most accurate ways to assess how and where we carry fat rely on complicated X-ray analysis. Some Google researchers have been working an easier way to offer that same sort of insight, using little more than the cameras on our phones. PhotoScan is Google Research's system for estimating body composition by using AI to analyze a series of 2D images of your body. First, Google trained the system using some of the data from those actual X-ray scans (a technique called dual-energy X-ray absorptiometry, or DXA), measuring both overall body fat levels, as well as ratios between core and waist/leg fat (A/G ratio), and organ and subcutaneous fat (V/S ratio). Combining that with body data from MRI scans, and then further fine-tuning it with a fresh set incorporating actual photos from smartphone cameras, the team's ultimate model demonstrated the ability to visually estimate body fat with a higher degree of accuracy than the measurements taken with tools like the bioelectrical impedance analysis (BIA) sensor on wearables. More than that, it showed high accuracy at estimating those A/G and V/S ratios, which BIA sensors can't even attempt. Beyond just attempting to offer insight into body composition, Google Research also wanted to see how PhotoScan results might be able to predict conditions like insulin resistance. That's been a very visible focus of Google's lately, like we see with the upcoming Insulin Resistance Trends in Health Guardian on the Pixel Watch. While the research looks pretty sound, it's one thing to announce that you've got a non-invasive, affordable new way to give users some of these useful body composition data points. But then you've also got to convince users to share a bunch of potentially unflattering photos with the PhotoScan tool, and even with the most robust privacy policy in the world, that could be a big ask. Then there's still the biggest problem of all: Even after showing them this data, how do you get users to actually make the serious diet and lifestyle changes they need? That's a very different, much more difficult issue, but hopefully tools like PhotoScan will ultimately make it easier to at least put us in a position to start addressing it. Right now Google isn't sharing any plans to turn PhotoScan into an actual product, but we wouldn't be surprised to see this tech pop up some day as a future Google Health offering.

Sorry, Samsung -- Google says its latest AI smartphone camera tool can measure bodies better than the Galaxy Watch's bioelectric sensors

Google's been quietly developing health scanning tools for your smartphone's camera * Google is building on its new Insulin Resistance Trends health feature with PhotoScan * PhotoScan is 'an investigational deep learning framework' that can estimate body fat using a phone's camera with 'near DXA accuracy' * It's not yet available to the public, but the research 'demonstrate[s] the feasibility' of such features As the Google Pixel Watch 5 launched last week alongside a slew of new Pixel phones, one of the most interesting new developments was the introduction of the Pixel Watch's new Health Guardian suite of tools, including Insulin Resistance Trends. Insulin Resistance Trends showcases a month-by-month breakdown containing estimations of how your body responded to blood sugar spikes -- a crucial component in weight management and the development of Type-2 diabetes. Body composition -- your ratio of different kinds of fat, skeletal muscle, and bone -- is also an excellent marker of insulin resistance and metabolic health, so body scans are often used to help determine insulin resistance. Body comp is often measured with medical-grade 'DXA' scans, which use X-Rays, or commercially available devices at home. These home devices can be the best smart scales or even some smartwatches, such as the Samsung Galaxy Watch Ultra 2. Without medical DXA scanners, they usually use a process called 'bioelectrical impedence analysis', or BIA. A low-level electric current is sent through the user's body, moving through fat, muscle, and bone at different speeds. The speed of the electrical current helps the device estimate the percentages of each material in the body: a useful thing to have on a health-monitoring device like a smartwatch. However, Google is busy researching an even more accurate version of a body scan without using any specialist equipment or BIA kit -- in fact, it's just a smartphone camera and an onboard AI model. This in-development feature is called PhotoScan, appropriately. According to its research blog, Google has been developing PhotoScan by matching smartphone photographs of subjects with medical-grade DXA scans, along with other subject information, then running it all through an AI model. The AI model eventually achieved 'strong DXA agreement' when estimating body fat percentage and other metrics, leading to 'near-DXA accuracy for predicting insulin resistance'. The researchers conclude that 'clinical DXA imaging delivers the most accurate body composition but lacks scalability, whereas wearable BIA sensors offer convenience but are limited to basic body fat percentage. 'Our PhotoScan approach offers a promising middle ground, estimating granular body composition from standard smartphone imagery with near-DXA accuracy.' Sorry, Samsung -- it looks like we'll be pivoting away from BIA sensors and towards smartphone cameras for body fat percentage calculation in future. While the technology doesn't seem to be commercially available yet, it does 'demonstrate the feasibility of smartphone-based body composition estimation,' and I'd be surprised not to see it make its way to the best Pixel phones eventually. Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.