Google Pixel Phones Revolutionize New York Subway Track Inspections with AI

New York's MTA strapped Google Pixels to moving subway cars, for science

In partnership with Google, the Metro Transit Authority took to tying several Google Pixel phones to subway cars in New York in order to track data. The experiment was part of the MTA's need to automate and expand track-safety inspections and repair, and tying a few Pixels to a train car seemed the best place to start. If you're into facts, the New York City subway carries somewhere over 3 million people per day, according to its own website. With that sort of volume, human-powered efforts to find, track defects, and maintain them can only be so efficient. It works, but there's room for improvement. The MTA is looking to supplement that effort with some sort of automation, and AI partnered with existing technology isn't so hard to get your hands on. According to WIRED, Google and the New York MTA partnered up to strap several devices to subway cars in order to listen for track defects while recording other movement data. Those devices weren't some specialized hardware for the professional sector; they were Pixels. The Google Public Sector worked with the New York City subway to provide several Pixels in an effort to experiment with existing, off-the-shelf hardware under operation TrackInspect. If TrackInspect was a success, it would showcase that everyday phones have the capability to provide enough data to supplement the work done by individuals in repairing and maintaining the rail system. The Pixels would need to collect audio, movement, and geographic data underground to be fed to AI training models that could efficiently package the data for repair teams. All of the sounds commuters take for granted -- the screeches and heavy crashes or bumps -- could be translated into a specific track that needs attention. While human inspection is still required, the goal is to automate most of the flagging system. Through all of the recordings of the New York subway that were made with the stowaway Pixel phones, 92% of recorded defects were corroborated by human inspectors. This project still used an inspector to listen to all of the collected audio and analyze vibration recordings, with an 80% success rate. The TrackInspect project collected 335 million sensor readings and 1,200 hours of audio. Those collections were used to train around 200 individual AI models for this exact work. The hope is that the MTA will be able to implement this technology further, possibly with specialized hardware instead of a device that was made to do something entirely different. The project proves that the tech available now can be implemented at little cost with the proper AI models involved.

MTA strapped Google Pixels to subway cars to spot track defects

The experiment found 92 percent of issues identified by inspectors. Anyone who has rode the New York City subway can tell you that it has a lot of problems, from strange noises to flammable debris on the tracks. Now, as is the solution for everything these days, the Metropolitan Transportation Authority (MTA) is testing how AI could improve the repair process with the help of six Google Pixel phones. In this case, the Google Pixel phones rode on four different subway cars between last September and January. The experiment, conducted in partnership with Google Public Sector, used the phone's accelerometers, magnetometers and microphones to pick up on any worrisome noises. This data was thn sent to cloud-based systems that generated predictive insights using machine learning algorithms. The tech, known by Google as TrackInspect, found 92 percent of the defect locations that inspectors located. "By being able to detect early defects in the rails, it saves not just money but also time -- for both crew members and riders" New York City Transit President Demetrius Crichlow stated in a release. "This innovative program -- which is the first of its kind -- uses AI technology to not only make the ride smoother for customers but also make track inspector's jobs safer by equipping them with more advanced tools." Typically, inspectors walk all 665 miles of the subway tracks to find any issues, along with sensor-laden "train geometry cars" picking up data three times a year. During the experiment, inspectors checked out any locations highlighted and confirmed whether there was a defect. They could also ask questions about maintenance and protocols through the tools generative AI system.

The New York City Subway Is Using Google Pixels to Listen for Track Defects

New York City's transit authority is one of a few US systems experimenting with using sensors and AI to improve track inspections. Between September and January, six Google Pixel smartphones hitched free rides on four New York City subway cars. Specifically, they took the A train, as it ping-ponged the 32 miles between the northern tip of Manhattan and the southern reaches of Queens. The phones weren't stowaways or strays, and an extremely sharp-eyed passenger could tell because they were inside plastic enclosures and secured via brackets to the cars' undersides and interiors. While people inside the cars used their smartphones to write emails or scroll Instagram or explore Roblox, the subway operators were using these phones' sensors -- accelerometers, magnetometers, and gyroscopes, and for those attached to the cars' exteriors, additional external microphones -- to listen intently. The phones were part of a brief experiment by New York City's Metropolitan Transportation Authority and Google into whether cheap, mostly off-the-shelf technology could supplement the agency's track inspection work. (Google Public Sector, the division that undertook the work, didn't charge the MTA for this initial experiment.) Today, inspections are carried out by human inspectors, who together walk all 665 miles of New York City's subway tracks, eyes peeled for problems like broken rails, busted signals, and water damage. Thrice-annual rides by specialized, sensor-laden "train geometry cars," also capture and upload more sophisticated data on the status of the city's rail infrastructure. New York City Transit's work with the experimental technology, which Google calls TrackInspect, suggests that audio, vibration, and location data, collected relatively cheaply and used to train artificial intelligence prediction models, can supplement that inspection work. It can point humans toward suspicious rattles, bangs, or squeals, suggesting what kinds of tools they'll need to make the repairs before they get there. Throughout the four-month project, the tech was able to identify 92 percent of the defect locations later pinpointed by human track inspectors, the MTA says. Eventually, the tech could become "a way we could minimize the amount of work that's done to identify those defects, and point inspectors in the right direction, so they can spend time fixing instead of identifying, and go directly there and do the work," says Demetrius Crichlow, the agency's president. In the future, the MTA hopes to create a "modernized" system that automatically identifies and organizes fixes for track issues. For the system's 3.7 million daily riders, catching defects before they become problems could be the difference between getting to work or school on time and getting mired in unexpected delays. "The goal with this [project] is to find issues before they become a major issue in terms of service," says Crichlow. The collaboration with Google will now expand to a full pilot project, the MTA says, where Google will build a production version of the tech and put it in the hands of track inspectors themselves. The Google experiment is part of a bumper crop of AI-enabled technology that transit agencies are just beginning to use to supplement their typical inspections, says Brian Poston, an assistant vice president of transit and rail with the consultancy WSP. While New York is unique in using "harmonics" -- audio and vibration -- to pinpoint issues, others have installed small sensors or cameras on tracks that take automated measurements and flag discrepancies as they emerge. The tech is enabled not just by advances in machine learning, but also cheaper and smaller batteries and processors. Still, US regulators require regular human inspection and maintenance of rails, and Poston says he doesn't expect those rules to go away anytime soon. "Until the technology can be specific and precise, you're always going to need that human interaction," he says.

Google's Pixel Phones Had A 92 Percent Success Rate When Detecting Problems In A Subway Car; These Could Be Used In The Future To Replace The Expensive Scanners

A few Pixel smartphones have been routinely used on four New York City subway cars since September of last year, and it was for a purpose that would not just save the service countless funds but also avoid potentially life-threatening situations. According to the latest report, the Metropolitan Transportation Authority (MTA) and Google were experimenting with how these devices could prevent problems in subway cars thanks to their advanced internal sensors, negating the need to install the significantly more expensive equipment. Traditionally, problems in subway cars or the infrastructure are inspected by humans, who have to walk 665 miles of the New York City transit while keeping their eyes open for anything unusual, such as broken rails or water damage. There are also 'train geometry cars' equipped with a bevy of sensors that capture and upload crucial data concerning the subway's infrastructure. However, what if there was an inexpensive way of carrying out all of these tasks? The Google Public Sector might be working on a solution. This division deployed experimental technology called TrackInspect, and when used by Pixel devices, can detect audio, vibration and location data. This information can then be used to train AI prediction models, which can go along way in assisting humans during the inspection work. During the four-month period, Google's handsets successfully identified 92 percent of the defect locations, which were later confirmed by the inspection team. This approach can become a stepping stone for something far more advanced, with MTA's President Demetrius Crichlow believing that a 'modernized' system can be created that identifies and initiates fixes for subways. As reported by Wired, Crichlow says that the goal is to identify the issues and eliminate them before they become a major problem and start disrupting the service. There are approximately 3.7 million daily commuters, meaning that these defects can become the difference for thousands of travellers who use the service for work or school. TrackInspect reportedly combined 335 million sensor readings and 1,200 hours of audio, along with New York City Transit's database of track defects, to train around 200 individual prediction models to find issues. The collaboration between MTA and Google will now transition to a full pilot project, with the Mountain View behemoth said to develop a production version that will be used by the track's inspection team. If successful, this project can lead to considerable cost savings.

Hear that? Google Pixels are listening for trouble on NYC subway tracks

Gemini's newest extension finally makes it a decent Google Assistant alternative Summary Smartphone sensors are powerful tools for monitoring and tracking your environment. Google and MTA collaborated on a project using Pixel phones to detect subway anomalies. Pixel phones located problem spots 92% of the time, showing promise for future subway monitoring. We've been living with smartphones for so long that it's easy to forget just how complex these devices really are. Smartphones are packed with a wide variety of sensors that can easily monitor and track you and the surrounding environment. So it makes perfect sense that a smartphone could be used as a monitoring tool. Related Smartphone sensors: An A-Z guide on your phone's many sensors How your phone knows it's in landscape mode Posts With that said, it looks like the Metropolitan Transportation Authority (MTA) and Google pulled off a brief experiment towards the end of 2024 that had Pixel phones listening for trouble in the subway tunnels of New York. That's right, a few Pixel phones were wired up outside train cars in order to capture as much data as possible in an effort to track possible anomalies that could lead to bigger issues. A new way to detect problems before they happen Wired reported on this project, dubbed TrackInspect, sharing that this experiment with Google makes use of technology that's readily and cheaply available in order to get the job done, or at least a part of it (via Android Authority). While this kind of work is normally done with specialized equipment and workers just walking the lines, making use of technology found in most phones, could make monitoring the subway lines easier if this method takes off. Most phones, including Pixel handsets, feature a set of complex sensors that can be used to capture information about the ambient light, audio, movement, vibrations, locations, and so much more. With this data, an AI model is trained, and it can find areas in the subways that might have potential issues. Furthermore, the AI model can also suggest what types of fixes may be necessary before people even go down into the tunnels. So you might be asking yourself, just how accurate is this technology? Well, the Wired piece highlights that Google's efforts were able to locate problem spots 92 percent of the time. Not bad for some Pixel phones that were strapped to the exterior of a subway car, right? Of course, this is just a test, and for now, the MTA will keep things as is, with workers and expensive equipment roaming the tunnels of the New York subway system in order to search for defects. With that said, there are plans to move forward with a full pilot. If this comes to fruition, Google will build custom devices using similar technology so that it can be used to scan the tracks, so it will be interesting to see just how much better the readings can be with this specialized equipment.