Hugging Face Expands LeRobot Platform with Massive Self-Driving Dataset

Hugging Face expands its LeRobot platform with training data for self-driving machines | TechCrunch

Last year, Hugging Face, the AI dev platform, launched LeRobot, a collection of open AI models, data sets, and tools to help build real-world robotics systems. On Tuesday, Hugging Face teamed up with AI startup Yaak to expand LeRobot with a training set for robots and cars that can navigate environments, like city streets, autonomously. The new set, called Learning to Drive (L2D), is over a petabyte in size, and contains data from sensors that were installed on cars in German driving schools. L2D captures camera, GPS, and "vehicle dynamics" data from driving instructors and students navigating streets with construction zones, intersections, highways, and more. There's a number of open self-driving training sets out there from companies including Alphabet's Waymo and Comma AI. But many of these focus on planning tasks like object detection and tracking, which require high-quality annotations, according to L2D's creators -- making them difficult to scale. In contrast, L2D is designed to support the development of "end-to-end" learning, its creators claim, which helps predict actions (e.g. when a pedestrian might cross the street) directly from sensor inputs (e.g. camera footage) "The AI community can now build end-to-end self-driving models," Yaak co-founder Harsimrat Sandhawalia and Remi Cadene, a member of the AI for robotics team at Hugging Face, wrote in a blog post. "L2D aims to be the largest open-source self-driving data set that empowers the AI community with unique and diverse 'episodes' for training end-to-end spatial intelligence." Hugging Face and Yaak plan to conduct real-world "closed-loop" testing of models trained using L2D and LeRobot this summer, deployed on a vehicle with a safety driver. The companies are calling on the AI community to submit models and tasks they'd like the models to be evaluated on, like navigating roundabouts and parking spaces.

Hugging Face's LeRobot Platform Gets New Dataset for AI-Powered Cars

The L2D dataset is more than 1PB in size L2D was collected from sensors installed on 60 EVs The sensors include cameras, GPS, IMUs, and more Hugging Face announced the expansion of its LeRobot platform on Wednesday with a large dataset aimed at automotive automation. The online artificial intelligence (AI) and machine learning (ML) repository said that the dataset was created in collaboration with the AI startup Yaak. Dubbed Learning to Drive (L2D), the dataset was collected from a suite of sensors installed on 60 electric vehicles (EVs) over a period of three years. The open-source dataset is aimed at enabling developers and the robotics community to build spatial intelligence solutions for the automobile industry. In a blog post, the company detailed the new AI dataset, calling it "the world's largest multimodal dataset aimed at building an open-sourced spatial intelligence for the automotive domain." The entire dataset is more than 1PB (one PetaByte) in size, and was collected using sensor suites installed on 60 EVs operated by driving schools in 30 German cities for three years. Identical sensors were used to ensure consistency in the data collected. The LeRobot platform was launched last year as a collection of open-source AI models, datasets, and accompanying tools that can help developers build AI-powered robotics systems. The policies in the dataset are divided into two groups of expert policies and student policies. The former is comprised of data from driving instructors while the latter comes from learner drivers. Hugging Face stated that the expert policy has zero driving mistakes and is considered optimal, whereas the student policy contains known sub-optimalities. Both groups include natural language instructions for driving tasks. Each group features all driving scenarios that are necessary for completion to obtain a driving licence in the European Union (EU). Some of these driving tasks include overtaking, roundabout handling, and track driving. Detailing the sensor suite used to capture the L2D data, Hugging Face said that each of the 60 Kia Niro EV models were equipped with six RGB cameras to capture the vehicle's surrounding in 360p, on-board GPS for vehicle location and mapping, an inertial measurement unit (IMU) to capture vehicle dynamics. All the data was captured with timestamps. Notably, the dataset is aimed at helping developers and robotics scientists build end-to-end self-driving AI models that can eventually be used to build fully autonomous vehicle systems. Hugging Face highlighted that the L2D dataset will be released in a phased manner, where each successive release will be a superset of the previous releases to ensure ease of access. The platform is also inviting the community to submit models for closed loop testing of the dataset with a safety driver. This will begin in summer 2025.