Runway's Act-One: Revolutionizing AI Character Animation with Human Performance

AI characters with a "human soul"? This is the revolutionary new tool from Runway - Softonic

Introducing Act-One: The tool that combines human acting with AI video generation Runway, the popular video generation platform with artificial intelligence, has introduced a new tool that promises to revolutionize character creation and cinema in general. It is Act-One, a technology capable of creating realistic animations of AI characters by capturing the movements of human actors. Act-One allows users to record themselves and then apply AI to completely modify their appearance without losing the original expressions or movements. According to Runway, this could solve one of the most evident problems of current technologies: the lack of realism in AI-generated performances. "With Act-One, eye lines, micro-expressions, rhythm, and speech patterns are faithfully represented in the final generated result," the company stated in its presentation. The new tool will also allow the creation of complex scenes by combining gen-3 AI video technology with human performance, opening new possibilities for content creators. "One of the model's strengths is the production of cinematic and realistic results from a large number of camera angles and focal lengths," added Runway on Twitter. Although it has not yet been widely tested, the initial examples shared by the company show that Act-One is very helpful in creating emotive and realistic performances, all without the need for motion capture suits or film cameras. Act-One promises to completely change the rules of the game in the field of AI-generated video, placing the human actor at the center of the action, but supported by the most advanced technology available. According to Runway, access to Act-One will begin gradually in the coming weeks, and it is expected to be available to everyone soon.

How AI can turn your home video into a Hollywood blockbuster

Want to star in an animated film as an anthropomorphic animal version of yourself? Runway's AI video creation platform has a new AI tool to do just that. The new Act-One feature may make motion-capture suits and manual computer animation unnecessary to match live action. Act-One streamlines what is usually a long process for facial animation. All you need is a video camera facing an actor and able to capture their face as they perform. The AI fueling Act-One reworks the facial movements and expressions from the inputted video to fit an animated character. Runway claims even the most nuanced emotions are visible through micro-expressions, eyeliners, and other facets of the performance. Act-One can even produce multi-character dialogue scenes, which Runway suggests are difficult for most generative AI video models. To produce one, a single actor performs multiple roles, and the AI animates the different performances mapped onto different characters in one scene as though they are talking to each other. That's a far cry from the laborious traditional animation requirements and makes animation far more accessible to creators with limited budgets or technical experience. Not that it's always going to match the skills of talented teams of animators with big movie budgets, but the relatively low barrier of entry might let amateurs and those with limited resources have the chance to play with character designs that are still realistic in portraying emotions, all without breaking the bank or missing deadlines. You can see some demonstrations below. Act-One is, in some ways, an enhancement for Runway's video-to-video feature within its Gen-3 Alpha model. But while that tool uses a video and a text prompt to adjust the setting, performers, or other elements, Act-One skips straight to mapping human expressions onto animated characters. It also fits with how Runway has been pushing out more features and options for its platform, such as the Gen-3 Alpha Turbo version of its model, which sacrifices some functionality for speed. Like its other AI video tools, Runway has some restrictions on Act-One to prevent people from misusing it or breaking its terms and conditions. You can't make content with public figures, for instance, and it employs techniques to ensure anyone whose voice is used in the final video has given their permission. The model is continuously monitored to spot any attempts to break those or other rules. "We're excited to see what forms of creative storytelling Act-One brings to animation and character performance. Act-One is another step forward in our goal to bringing previously sophisticated techniques to a broader range of creators and artists," Runway wrote in its announcement. "We look forward to seeing how artists and storytellers will use Act-One to bring their visions to life in new and exciting ways." Act-One may be somewhat unique among AI video generators, though Adobe Firefly and Meta's MovieGen have some similar efforts in their portfolio. Runway's Act-One seems to be much easier to use than Firefly's equivalent and more available than the restricted MovieGen model.

Runway just changed filmmaking forever -- Act-1 lets you control AI characters

Runway, one of the leading artificial intelligence video platforms, has just announced a new feature that will completely change the game for character consistency and filmmaking in general. Act-1 is a new approach to AI video generators. It is a form of modern-day puppeteering, allowing you to film yourself or an actor performing a part and then use AI to completely change the way they look. This solves one of the biggest problems with AI -- consistency. Access to Act-1 will begin gradually rolling out over the coming weeks. Runway says it will soon be available to everyone. AI video tools are getting much better at human motion, lip-synching and character development, but they have a way to go before they can bridge the 'obviously AI' gap. Runway's new tool may have finally solved that problem. Instead of leaving the AI to work out how the character should move or react, it lets you upload a video along with a control image (to set the style) and basically maps the control image over your performance. For me, the true benefit of AI video will come from the merger of real and generative AI rather than relying completely on AI itself. The best films already make use of visual effects alongside model shots and film shots, and artificial intelligence is just an extension of that. Runway's Act-1 puts human performance front and center, using AI as an overlay. You're essentially turning the human into the puppet master, a bit like Andy Serkis and his performance of Gollum in "Lord of the Rings" -- only without the need for motion capture suits and expensive cameras. I haven't had the chance to try it yet, but judging by some of the examples shared by Runway, it's as simple as sitting in front of a camera and moving your head around. An element of this has already been available for some time, including from Adobe, but without the generative AI element. But it goes much further than we've seen in any tools so far. According to Runway: "With Act-1, eye-lines, micro-expressions, pacing and delivery are all faithfully represented in the final generated output." It also goes beyond simple puppeteering as Act-1 can create complex scenes using existing gen-3 AI video technology and integrate them into human performance. The company explained on X: "One of the model's strengths is producing cinematic and realistic outputs across a robust number of camera angles and focal lengths, allowing you to generate emotional performances with previously impossible character depth, opening new avenues for creative expression." Access to Act-1 will begin gradually rolling out to users today and will soon be available to everyone.

Runway's Mindblowing Act-One Transforms an Actor into a Cartoon From Just One Video

The AI video platform Runway has unveiled a remarkable new tool that transforms a person into a computer-generated character. Called Act-One, it takes a video of someone talking -- which can be shot on just a smartphone -- and uses the performance as an input to create compelling animations. Traditionally, this type of technology -- transposing an actor's performance onto a cartoon used in films like Avatar -- is complex involving motion capture equipment, facial capture rigging, and multiple footage reference. But Runway says it's their mission to "build expressive and controllable tools for artists that can open new avenues for creative expression." "The key challenge with traditional approaches lies in preserving emotion and nuance from the reference footage into the digital character," Runway writes in a blog post. "Our approach uses a completely different pipeline, driven directly and only by the performance of an actor and requiring no extra equipment." Act-One can be applied to a wide variety of reference images including cartoons and realistic-looking computer-generated humans, essentially deepfakes. "The model also excels in producing cinematic and realistic outputs, and is remarkably robust across camera angles while maintaining high-fidelity face animations," says Runway. "This capability allows creators to develop believable characters that deliver genuine emotion and expression, enhancing the viewer's connection to the content." Runway says that users will be able to create high-quality, narrative content using nothing more than a consumer-grade camera and one actor reading lines. The actor can even play different characters. Act-One has begun rolling out to Runway users and will soon be available to everyone.

'This is a game changer': Runway releases new AI facial expression motion capture feature Act-One

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More AI video has come incredibly far in the years since the first models debuted in late 2022, increasing in realism, resolution, fidelity, prompt adherence (how well they match the text prompt or description of the video that the user typed) and number. But one area that remains a limitation to many AI video creators -- myself included -- is in depicting realistic facial expressions in AI generated characters. Most appear quite limited and difficult to control. But no longer: today, Runway, the New York City-headquartered AI startup backed by Google and others, announced a new feature "Act-One," that allows users to record video of themselves or actors from any video camera -- even the one on a smartphone -- and then transfers the subject's facial expressions to that of an AI generated character with uncanny accuracy. The free-to-use tool is gradually rolling out "gradually" to users starting today, according to Runway's blog post on the feature. While anyone with a Runway account can access it, it will be limited to those who have enough credits to generate new videos on the company's Gen-3 Alpha video generation model introduced earlier this year, which supports text-to-video, image-to-video, and video-to-video AI creation pipelines (e.g. the user can type in a scene description, upload an image or a video, or use a combination of these inputs and Gen-3 Alpha will use what its given to guide its generation of a new scene). Despite limited availability right now at the time of this posting, the burgeoning scene of AI video creators online is already applauding the new feature. As Allen T. remarked on his X account "This is a game changer!" It also comes on the heels of Runway's move into Hollywood film production last month, when it announced it had inked a deal with Lionsgate, the studio behind the John Wick and Hunger Games movie franchises, to create a custom AI video generation model based on the studio's catalog of more than 20,000 titles. Simplifying a traditionally complex and equipment-heavy creative proccess Traditionally, facial animation requires extensive and often cumbersome processes, including motion capture equipment, manual face rigging, and multiple reference footages. Anyone interested in filmmaking has likely caught sight of some of the intricacy and difficulty of this process to date on set or when viewing behind the scenes footage of effects-heavy and motion-capture films such as The Lord of the Rings series, Avatar, or Rise of the Planet of the Apes, wherein actors are seen covered in ping pong ball markers and their faces dotted with marker and blocked by head-mounted apparatuses. Accurately modeling intricate facial expressions is what led David Fincher and his production team on The Curious Case of Benjamin Button to develop whole new 3D modeling processes and ultimately won them an Academy Award, as reported in a prior VentureBeat report. Yet in the last few years, new software and AI-based startups such as Move have sought to reduce the equipment necessary to perform accurate motion capture -- though that company in particular has concentrated primarily on full-body, more broad movements, whereas Runway's Act-One is focused more on modeling facial expressions. With Act-One, Runway aims to make this complex process far more accessible. The new tool allows creators to animate characters in a variety of styles and designs, without the need for motion-capture gear or character rigging. Instead, users can rely on a simple driving video to transpose performances -- including eye-lines, micro-expressions, and nuanced pacing -- onto a generated character, or even multiple characters in different styles. As Runway wrote on its X account: "Act-One is able to translate the performance from a single input video across countless different character designs and in many different styles." The feature is focused "mostly" on the face "for now," according to Cristóbal Valenzuela, co-founder and CEO of Runway, who responded to VentureBeat's questions via direct message on X. Runway's approach offers significant advantages for animators, game developers, and filmmakers alike. The model accurately captures the depth of an actor's performance while remaining versatile across different character designs and proportions. This opens up exciting possibilities for creating unique characters that express genuine emotion and personality. Cinematic realism across camera angles One of Act-One's key strengths lies in its ability to deliver cinematic-quality, realistic outputs from various camera angles and focal lengths. This flexibility enhances creators' ability to tell emotionally resonant stories through character performances that were previously hard to achieve without expensive equipment and multi-step workflows. The tool's ability to faithfully capture the emotional depth and performance style of an actor, even in complex scenes. This shift allows creators to bring their characters to life in new ways, unlocking the potential for richer storytelling across both live-action and animated formats. While Runway previously supported video-to-video AI conversion as previously mentioned in this piece, which did allow users to upload footage of themselves and have Gen-3 Alpha or other prior Runway AI video models such as Gen-2 "reskin" them with AI effects, the new Act-One feature is optimized for facial mapping and effects. As Valenzuela told VentureBeat via DM on X: "The consistency and performance is unmatched with Act-One." Enabling more expansive video storytelling A single actor, using only a consumer-grade camera, can now perform multiple characters, with the model generating distinct outputs for each. This capability is poised to transform narrative content creation, particularly in indie film production and digital media, where high-end production resources are often limited. In a public post on X, Valenzuela noted a shift in how the industry approaches generative models. "We are now beyond the threshold of asking ourselves if generative models can generate consistent videos. A good model is now the new baseline. The difference lies in what you do with the model -- how you think about its applications and use cases, and what you ultimately build," Valenzuela wrote. Safety and protection for public figure impersonations As with all of Runway's releases, Act-One comes equipped with a comprehensive suite of safety measures. These include safeguards to detect and block attempts to generate content featuring public figures without authorization, as well as technical tools to verify voice usage rights. Continuous monitoring also ensures that the platform is used responsibly, preventing potential misuse of the tool. Runway's commitment to ethical development aligns with its broader mission to expand creative possibilities while maintaining a strong focus on safety and content moderation. Looking ahead As Act-One gradually rolls out, Runway is eager to see how artists, filmmakers, and other creators will harness this new tool to bring their ideas to life. With Act -ne, complex animation techniques are now within reach for a broader audience of creators, enabling more people to explore new forms of storytelling and artistic expression. By reducing the technical barriers traditionally associated with character animation, the company hopes to inspire new levels of creativity across the digital media landscape.

Runway Act-One generates animations from video and voice inputs

Runway has announced the release of its latest tool, Act-One, designed to enhance character animation with greater realism and expressiveness. This new addition to the Gen-3 Alpha suite marks a significant advancement in how generative models are used for creating live action and animated content. Traditionally, creating facial animations requires complex workflows involving motion capture, manual face rigging, and multiple footage references. These methods often aim to capture and replicate the actor's emotions in a digital character. However, the challenge lies in preserving the original emotion and nuance of the performance. With Act-One, Runway introduces a streamlined process. The tool generates animations directly from an actor's video and voice performance, removing the need for additional equipment like motion capture devices. This simplification makes it easier for creators to animate characters without compromising the expressiveness of the original performance. Act-One is versatile, allowing creators to apply animations to a wide variety of reference images, regardless of the proportions of the source video. It can accurately translate facial expressions and movements into characters that may differ in size and shape from the original. This opens new doors for inventive character design, particularly in fields like animated content creation. The tool also shines in live-action settings, producing cinematic, realistic outputs that maintain fidelity across different camera angles. This functionality helps creators develop characters that resonate with viewers by delivering genuine emotion and expression, strengthening the connection between audience and content. Runway bets big on AI with $5 million fund for experimental films Runway is positioning Act-One as a solution for creating expressive dialogue scenes that were previously difficult to achieve with generative models. With only a consumer-grade camera and a single actor, creators can now generate scenes involving multiple characters, each portrayed with emotional depth. "Our approach uses a completely different pipeline, driven directly and only by a performance of an actor and requiring no extra equipment," Runway said in its blog post, highlighting the tool's focus on ease of use for creators. Runway remains committed to ensuring its tools are used responsibly. Act-One comes with a range of safety features, including measures to detect and block attempts to create content featuring public figures. Additional protections include verifying that users have the rights to the voices they create using Custom Voices and continuously monitoring for potential misuse of the platform. "As with all our releases, we're committed to responsible development and deployment," the company stated. The Foundations for Safe Generative Media serve as the basis for these safety measures, ensuring that the tool's potential is used in a secure, ethical way. With the gradual rollout of Act-One starting today, Runway aims to make advanced animation tools more accessible to a wider range of creators. By removing barriers to entry and simplifying the animation process, the company hopes to inspire new forms of creative storytelling. "Act-One is another step forward in our goal to bringing previously sophisticated techniques to a broader range of creators and artists," Runway emphasized.

Runway's Act-One Simplifies Character Animation for Creators

Runway, an NYC based AI video startup, announced Act-One, a new state-of-the-art tool for generating expressive character performances, inside Gen-3 Alpha. The access to Act One is currently limited. Act-One can generate compelling animations by just using video and voice performances as inputs. The tool reduces the reliance on traditional motion capture systems, making it simpler to bring characters to life in production workflows. On their blog, Runway uploaded several videos and styles showcasing the different ways in which the tool is used. Act-One simplifies animation by using a single-camera setup to capture actor performances, eliminating the need for motion capture or complex rigging. The tool preserves realistic facial expressions and adapts performances to characters of different proportions. The model delivers high-fidelity animations across various camera angles and supports both live-action and animated content. It expands creative boundaries for professionals as they require only consumer-grade equipment to produce expressive multi-turn dialogue scenes with a single actor. "Traditional pipelines for facial animation often involve complex, multi-step workflows. These can include motion capture equipment, multiple footage references, manual face rigging, among other techniques. Our approach uses a completely different pipeline, driven directly and only by the performance of an actor and requiring no extra equipment," per a statement on their blog. Last month, Runway partnered with Lionsgate to introduce AI into filmmaking. Runway aims to bring these tools for artists, and by extension bring their stories to life. This deal would open the doors for many of these stories to appear on the big screen eventually. Runway's tools have also been employed in Hollywood before. "I don't think text prompts are here to stay for a long time. So a lot of our innovation has been on creating control tools," said Runway CEO Cristóbal Valenzuela in an interview on how AI is coming to Hollywood and the need for giving creators more access and freedom over video generation. Runway also has an AI Film Festival which is dedicated to celebrating artists who incorporate emerging AI techniques in their short films. Launched two years ago, the festival aims to spark conversation about the growing influence of AI tools in the film industry and to engage with creators from diverse backgrounds, exploring their insights and perspectives. OpenAI's flagship model, Sora, is not publicly available yet, nor is there any update from the company regarding its release but hopefully it will be launched after elections. Genmo also unveiled a research preview of Mochi 1 - an open-source model designed to generate high-quality videos from text prompts. Earlier this month, Meta also entered the Gen AI space with its MovieGen. Adobe also introduced Gen AI to video with its Adobe Firefly. Luma's Dream Machine was made freely available for experimentation on its website. In terms of competition from China, Minimax officially launched its Image-to-Video feature. Even Kling introduced new capabilities to its model including a lip sync feature.

Runway Act-One's Latest Gen-3 Alpha Model Introduces AI-Driven Facial Expression Capture

This feature enables precise capture and application of facial expressions to AI-generated characters, revolutionizing AI video generation by simplifying facial animation Runway AI, a company known for its innovations in video generation technology, has announced a new feature, Act-One, as part of its Gen-3 Alpha large language model (LLM). This cutting-edge tool enables accurate capture and reproduction of facial expressions from a source video, making it possible to apply those expressions to AI-generated characters in videos. The introduction of Act-One marks a significant advancement in AI video generation, addressing one of the key limitations: replicating realistic expressions in AI-generated characters. Facial animation has long been a complicated affair requiring multi-step workflows like manual face rigging, motion capture and shooting the actor from several angles or various points of view. Runway AI's Act-One Changes the game as users can record a single-point video of either themselves or an actor in a bid to capture their movements, eye focus and micro-expressions from the mobile device. This tool allows the user, and other performers, to only provide minimal input; micro-facial expressions, eye focus and the character will incorporate these recordings. These performances are then incorporated into AI characters regardless of the proportions or angle that the actual video displays to the source video. This instrument applies to both real-world figures and animated figures providing users the possibility to create various genres of videos from movie scenes to cartoons. Runway AI is worth mentioning the flexibility of the tool: "In addition, the model retains valuable imitation and accurately refers to action with the character whose body shape is different from the source video context." Such a wide range of applications is expected to open new frontiers of creativity in character design and animation such that creators will be able to come up with quality and more emotive material with ease. Currently, Act-One is introduced to users and the gradual rollout for free account holders with a capped amount of video-generating tokens allows them to utilize the tool. The feature is available only for the Gen-3 Alpha model developed by Runway and seeks to enhance the realism in AI-generated footage by providing an easier method of animating facial movements and gestures which does not require complicated machinery or processes to perform. With this, it can be said that the Act-One introduction to Gen-3 Alpha model by Runway AI is a major advancement in video generation using AI algorithms. Act-One technology reduces the complexity involved in facial animation, thus enabling the creators to create more lifelike and expressive AI models with ease. Besides resolving important problems in the video content creation process, this tool will open new horizons for animated and live-action storytelling which combines these techniques. As the feature expands to more creators it will be a game changer as the world has never seen quality AI videos for the masses.

Runway Can Now Add Your Facial Expression to an AI Character

The feature can generate both animated and realistic videos Runway AI, an artificial intelligence (AI) firm focusing on video generation models, announced a new feature on Tuesday. Dubbed Act-One, the new capability is available within the company's latest Gen-3 Alpha large language model (LLM) and is said to accurately capture facial expressions from a source video and then reproduce them on an AI-generated character in a video. The feature solves a significant pain point in AI video generation technology which is converting real people into AI characters while not losing out on realistic expressions. In a blog post, the AI firm detailed the new video generation capability. Runway stated that the Act-One tool can create live-action and animated content using video and voice performances as inputs. The tool is aimed at offering expressive character performance in AI-generated videos. AI-generated videos have changed the video content creation process significantly as individuals can now generate specific videos using text prompts in natural language. However, there are certain limitations that have prevented the adaptation of this technology. One such limitation is the lack of controls to change the expressions of a character in a video or to improve their performance in terms of delivery of a sentence, gestures, and eye movement. However, with Act-One, Runway is trying to bridge that gap. The tool, which only works with the Gen-3 Alpha model, simplifies the facial animation process, which can often be complex and require multi-step workflows. Today, animating such characters requires recording videos of an individual from multiple angles, manual face rigging, and capturing their facial motion separately. Runway claims Act-One replaces the workflow and turns it into a two-step process. Users can now record a video of themselves or an actor from a single-point camera, which can also be a smartphone, and select an AI character. Once done, the tool is claimed to faithfully capture not only facial expressions but also minor details such as eye movements, micro-expressions as well as the style of delivery. Highlighting the scope of this feature, the company stated in the blog post, "The model preserves realistic facial expressions and accurately translates performances into characters with proportions different from the original source video. This versatility opens up new possibilities for inventive character design and animation." Notably, while Act-One can be used for animated characters, it can also be used for live-action characters in a cinematic sequence. Further, the tool can also capture details even if the angle of the actor's face is different from the angle of the AI character's face. This feature is currently being rolled out to all users gradually, however, since it only works with Gen-3 Alpha, those on the free tier will get a limited number of tokens to generate videos with this tool.

Runway's Act-One uses smartphone cameras to replicate facial expression motion capture - SiliconANGLE

Runway's Act-One uses smartphone cameras to replicate facial expression motion capture Runway AI Inc., the generative artificial intelligence startup that builds tools for AI-generated video creation, has announced the launch of a new feature to help creators give their AI video characters more realistic facial expressions. It's called Act-One, and it makes it possible for existing users to record themselves on something as simple as a smartphone camera, capture their facial expressions, and then replicate them on an AI-generated video character. Runway said in a blog post that the tool is being rolled out to users starting from today, and can be accessed by anyone with a Runway account. That said, it's not entirely free-to-use, as users will be required to have enough credits on their account to access the startup's most advanced Gen-3 Alpha video generation model. The Gen-3 Alpha model debuted earlier this year, introducing support for text-to-video, image-to-video and video-to-video modalities, meaning that users can write a description of a scene, upload an image or a video, or use a combination of those inputs as prompts. Once prompted, the model will go about creating a slick video that tries to match the user's vision. Although Runway's Gen-3 Alpha model can create some impressive videos, one area where it has always been a bit weak is facial animation. Particularly, creating accurate facial expressions on characters that can match the mood of the scene. In the filmmaking industry, facial animation is an intricate and expensive task that involves using sophisticated motion capture technologies, manual face rigging techniques and lots of heavy editing behind-the-scenes. Runway is trying to make advanced facial animation more accessible with Act-One. Using the tool, creators will be able to animate their video characters in almost any way they can imagine, without needing to use pricey motion capture equipment. Instead, Act-One makes it possible to use your own videos and facial expressions as a kind of reference, transposing them onto AI-generated characters. It's incredibly detailed, able to replicate everything from micro-expressions to eye-lines, onto various different characters. In a post on its official X account, Runway said Act-One can "translate the performance from a single input video across countless character designs and in many different styles". Although it has not yet rolled out to every Runway user, the company has already received positive feedback from creators: Act-One can be utilized by a range of creative professionals, including animators, video game developers and indie filmmakers, enabling them to generate more unique characters whose personality and actions can be reflected with their emotions and expressions. They'll be able to create much more realistic, cinema-like video characters and capture them at any camera angle or focal length, Runway said, unlocking the potential for much richer, more detailed storytelling and artistic expression. By eliminating the technical barrier associated with character animation, the company hopes to inspire a new generation of creators to better express themselves. For instance, it means an indie film producer can use a single actor to take on the role of multiple animated characters that display Hollywood-level realism, using only a consumer-grade camera. In a post on X, Runway co-founder and Chief Executive Cristóbal Valenzuela said the filmmaking industry is becoming much more receptive to the potential of generative AI: Runway added that Act-One comes with a number of built-in safeguards that will prevent misuse, including guardrails that will prevent efforts to generate content featuring public figures without their express authorization. It's also said to be integrated with tools to verify voice usage rights, the company said. In addition, Runway will perform continuous monitoring of the tool to ensure it is used in a responsible way by creators.