AI-Powered De-Aging Technology Revolutionizes Filmmaking in Robert Zemeckis' "Here"

New AI-powered tech allows real-time de-aging in Tom Hanks and Robin Wright's latest movie

In brief: The use of AI in movies remains a contentious subject, but there are some instances where using the technology seems more acceptable than others. Digitally de-aging actors, for example, has been a common sight on the big screen for years, and never has the process been more advanced than in Here, the new Robert Zemeckis film starring Tom Hanks and Robin Wright. Based on the 2014 graphic novel, Here is filmed from a single camera position across time, from when dinosaurs roamed the earth to the modern era. The story follows Hanks' and Wright's characters from their teenage years to old age, which means AI-powered effects feature heavily. Turning 68-year-old Hanks and 58-year-old Wright into young versions of themselves was the work of visual effects company Metaphysic. The firm's facial modification system works by training machine-learning models on frames from Hanks' and Wright's previous films, showing them at various ages, angles, and in different lighting. One imagines that The Princess Bride and The Man with One Red Shoe were part of that library. Unlike traditional de-aging effects that require extensive and lengthy post-production work, the Metaphysic Live VFX tech instantly applies the de-aging effect to actors during filming by analyzing facial landmarks and applying the de-aged faces, hence the "Live" part of the name. The use of Metaphysic Live meant that the film crew could watch two monitors during the filming of Here: one displaying the unaltered footage of the actors, the other showing them at different ages of their lives. What Hanks really looked like at 29 "This technology doesn't require dots on faces. It doesn't require multiple witness cameras or any other sort of intrusive technology," VFX supervisor Kevin Baillie told Variety. Around 40 minutes of Here's runtime features de-aged actors, including Kelly Reilly and Paul Bettany, who play Hank's character's parents, yet it still managed to stay within its comparatively low $45 million budget. Using modern CGI to digitally de-age movie stars has seen huge advances in recent years - just look at the very unnatural looking "young" Jeff Bridges in 2010's Tron: Legacy compared to today's efforts, such as Robert De Niro in The Irishman and Harrison Ford in Indiana Jones and the Dial of Destiny. While de-aging stars seems to be one of the less controversial ways of using AI in the film industry, its ability to bring actors back from the dead is more of a hot-button topic. In September, it was reported that a company was suing Disney over its recreation of deceased actor Peter Cushing in Rogue One. Robert Downey Jr., meanwhile, recently warned that his estate will sue all future executives who recreate his likeness after his death.

New Zemeckis film used AI to de-age Tom Hanks and Robin Wright

On Friday, TriStar Pictures released Here, a $50 million Robert Zemeckis-directed film that used real-time generative AI face transformation techniques to portray actors Tom Hanks and Robin Wright across a 60-year age span, marking one of Hollywood's first full-length features built around AI-powered visual effects. The film adapts a 2014 graphic novel set primarily in a New Jersey living room across multiple time periods. Rather than cast different actors for various ages, the production used AI to modify Hanks' and Wright's appearances throughout. The de-aging technology comes from Metaphysic, a visual effects company that creates real-time face swapping and aging effects. During filming, the crew watched two monitors simultaneously: one showing the actors' actual appearances and another displaying them at whatever age the scene required. Metaphysic developed the facial modification system by training custom machine-learning models on frames of Hanks' and Wright's previous films. This included a large dataset of facial movements, skin textures, and appearances under varied lighting conditions and camera angles. The resulting models can generate instant face transformations without the months of manual post-production work traditional CGI requires. Unlike previous aging effects that relied on frame-by-frame manipulation, Metaphysic's approach generates transformations instantly by analyzing facial landmarks and mapping them to trained age variations. "You couldn't have made this movie three years ago," Zemeckis told The New York Times in a detailed feature about the film. Traditional visual effects for this level of face modification would reportedly require hundreds of artists and a substantially larger budget closer to standard Marvel movie costs. This isn't the first film that has used AI techniques to de-age actors. ILM's approach to de-aging Harrison Ford in 2023's Indiana Jones and the Dial of Destiny used a proprietary system called Flux with infrared cameras to capture facial data during filming, then old images of Ford to de-age him in post-production. By contrast, Metaphysic's AI models process transformations without additional hardware and show results during filming.

'Amazing' AI de-ages Tom Hanks in new film 'Here'

LOS ANGELES, United States (AFP) - Tom Hanks has praised the "amazing" use of artificial intelligence to de-age him "in real time" on the set of new movie "Here," even as he accepted that the technology is causing huge concern in Hollywood. "Here," out in theatres Friday, stars Hanks and Robin Wright as a couple striving to keep their family together through births, marriages, divorces and deaths, across multiple decades and even generations. Hanks portrays his character from an idealistic teen, through various stages of youth and middle age, to a frail, elderly man. But rather than just relying on makeup, filmmakers teamed up with AI studio Metaphysic on a tool called Metaphysic Live, to rejuvenate and "age up" the actors. The technology worked so fast that Hanks was able to immediately watch his "deep-faked" performance after each scene. "The thing that is amazing about it is it happened in real time," said Hanks. "We did not have to wait for eight months of post-production. There were two monitors on the set. One was the actual feed from the lens, and the other was just a nanosecond slower, of us 'deep-faked.' "So we could see ourselves in real time, right then and there." The rapidly increasing use of AI in films including "Here" has triggered vast concern in Hollywood, where actors last year went on strike over, among other things, the threat they believe the technology poses to their jobs and industry. Hanks acknowledged those fears during a panel discussion with director Robert Zemeckis at last weekend's AFI Fest in Hollywood, saying a "lot of people" were worried about how it will be used. "They took 8 million images of us from the web. They scraped the web for photos of us in every era that we've ever been - every event we've filmed, every movie still, every family photo that might have existed anywhere," Hanks explained. "And they put that into the box - what is it, 'deepfake technology,' whatever you want to call it." 'Cinematic' The use of AI is not the only unusual technological feat in "Here." The film is entirely shot from one static camera, positioned for the most part in the corner of a suburban US home's living room. Viewers occasionally see glimpses of the same geographic space before the house was built, as the action hops back and forth to colonial and pre-colonial times - or even earlier. "Here" is based on a graphic novel by Richard McGuire, which uses the same concept. "It had to be true to the style of the book, and that's why it looks the way it does," Zemeckis told AFP. "It worked in levels that I didn't expect. It's got a real powerful intimacy to it, and in a wonderful way, it's very cinematic." But the film's use of AI has drawn the most attention. 'Very serious subject' AI was also at the heart of a very different film at AFI Fest - "Wallace & Gromit: Vengeance Most Fowl," the latest film for the beloved British stop-motion characters. When Wallace constructs a "smart gnome" to take care of chores, his faithful pooch Gromit immediately sniffs danger. Once Feathers McGraw - the nefarious penguin introduced to audiences in 1993 short film "The Wrong Trousers" - gets involved, the technology takes a sinister turn. AI becomes "the wedge between Wallace and Gromit," explained co-director Merlin Crossingham. "It is a very light touch, although it's a very serious subject," he said. If "we can trigger some more intellectual conversation from our silly adventure with Wallace and Gromit, then that can't be a bad thing." The film itself did not use AI. "We don't and we wouldn't," said Crossingham, earning hearty applause from the Hollywood crowd.

'Amazing' AI de-ages Tom Hanks in new film 'Here'

Los Angeles (United States) (AFP) - Tom Hanks has praised the "amazing" use of artificial intelligence to de-age him "in real time" on the set of new movie "Here," even as he accepted that the technology is causing huge concern in Hollywood. "Here," out in theaters Friday, stars Hanks and Robin Wright as a couple striving to keep their family together through births, marriages, divorces and deaths, across multiple decades and even generations. Hanks portrays his character from an idealistic teen, through various stages of youth and middle age, to a frail, elderly man. But rather than just relying on makeup, filmmakers teamed up with AI studio Metaphysic on a tool called Metaphysic Live, to rejuvenate and "age up" the actors. The technology worked so fast that Hanks was able to immediately watch his "deep-faked" performance after each scene. "The thing that is amazing about it is it happened in real time," said Hanks. "We did not have to wait for eight months of post-production. There were two monitors on the set. One was the actual feed from the lens, and the other was just a nanosecond slower, of us 'deep-faked.' "So we could see ourselves in real time, right then and there." The rapidly increasing use of AI in films including "Here" has triggered vast concern in Hollywood, where actors last year went on strike over, among other things, the threat they believe the technology poses to their jobs and industry. Hanks acknowledged those fears during a panel discussion with director Robert Zemeckis at last weekend's AFI Fest in Hollywood, saying a "lot of people" were worried about how it will be used. "They took 8 million images of us from the web. They scraped the web for photos of us in every era that we've ever been -- every event we've filmed, every movie still, every family photo that might have existed anywhere," Hanks explained. "And they put that into the box -- what is it, 'deepfake technology,' whatever you want to call it." 'Cinematic' The use of AI is not the only unusual technological feat in "Here." The film is entirely shot from one static camera, positioned for the most part in the corner of a suburban US home's living room. Viewers occasionally see glimpses of the same geographic space before the house was built, as the action hops back and forth to colonial and pre-colonial times -- or even earlier. "Here" is based on a graphic novel by Richard McGuire, which uses the same concept. "It had to be true to the style of the book, and that's why it looks the way it does," Zemeckis told AFP. "It worked in levels that I didn't expect. It's got a real powerful intimacy to it, and in a wonderful way, it's very cinematic." But the film's use of AI has drawn the most attention. 'Very serious subject' AI was also at the heart of a very different film at AFI Fest -- "Wallace & Gromit: Vengeance Most Fowl," the latest film for the beloved British stop-motion characters. When Wallace constructs a "smart gnome" to take care of chores, his faithful pooch Gromit immediately sniffs danger. Once Feathers McGraw -- the nefarious penguin introduced to audiences in 1993 short film "The Wrong Trousers" -- gets involved, the technology takes a sinister turn. AI becomes "the wedge between Wallace and Gromit," explained co-director Merlin Crossingham. "It is a very light touch, although it's a very serious subject," he said. If "we can trigger some more intellectual conversation from our silly adventure with Wallace and Gromit, then that can't be a bad thing." The film itself did not use AI. "We don't and we wouldn't," said Crossingham, earning hearty applause from the Hollywood crowd. "Vengeance Most Fowl" will be broadcast on Christmas Day in the United Kingdom and Ireland, before airing globally on Netflix from January 3.