Meta Unveils Movie Gen: A Groundbreaking AI Video and Audio Creation Tool

Meta's Movie Gen Is Latest in eCommerce Content Creation Tools | PYMNTS.com

Meta's new Movie Gen artificial intelligence (AI) video creation tool could level the playing field for small businesses in the digital marketplace, experts say. Movie Gen joins an expanding list of AI-powered video generation tools. In recent months, several major players have introduced or enhanced their offerings in this space. Industry analysts predict Movie Gen's entry will intensify competition in the eCommerce sector, slashing content production costs for businesses of all sizes. "Personalized videos have emerged as the most effective tool in cultivating a loyal customer base, with 64% of consumers more likely to make a purchase after watching a personalized video," Ryan McDonald, COO of Resell Calendar, an eCommerce platform for resellers, told PYMNTS. He sees AI-powered video generation as a game-changer for small businesses, allowing them to compete with larger corporations in creating targeted content at scale. McDonald said, "With the power of this tool, independent business owners can generate specific videos based on customers' preferences and/or purchase history with much less time compared to doing it all by themselves." This efficiency allows retailers to "spend more time thinking about the ideas of these videos," whether they're for "new releases, special offers, product recommendations, or just a simple 'Happy Birthday!'" Movie Gen can generate content from user prompts, edit existing videos, and add synchronized sound effects and background music. While the company has no plans to release it for open developer use, it intends to collaborate with the entertainment industry and content creators. The announcement comes amid both excitement and concern in Hollywood over AI's potential impact on production and copyright issues. The options for using AI to create videos are expanding. These tools vary in their specific capabilities, output quality, and ease of use, but all share the common goal of democratizing video content creation. They offer features ranging from text-to-video generation to video editing and extension, with some focusing on photorealism while others excel in animation or stylized content. OpenAI's Sora, announced earlier this year can create highly realistic videos up to one minute long from text prompts, showcasing advanced capabilities in motion, physics and scene composition. Google's Lumiere specializes in creating short video clips focusing on realistic motion. Runway, a startup that has gained traction, released Gen-2 in March 2023. This tool allows users to generate, edit and extend videos using text prompts or image inputs. Stability AI, known for its image generation model Stable Diffusion, introduced Stable Video Diffusion, capable of transforming still images into short video clips. Another emerging player, Pika Labs, launched its eponymous tool which excels at creating animations and visual effects. Meanwhile, Nvidia's Vid2Vid technology, while not a standalone product, demonstrates the potential for AI to transform existing video content based on style transfer or scene manipulation. As platforms like TikTok gain prominence as search engines, particularly among younger people, AI-generated video content could become vital for businesses looking to maintain visibility and engagement. This shift may compel traditional content creators and marketing agencies to adapt to an AI-driven commerce or risk falling behind more tech-savvy competitors. AI-generated media is likely to become more sophisticated and accessible, and experts foresee changes in advertising strategies, product demonstrations, and customer experiences across digital storefronts. AI text-to-video tools allow businesses to rapidly create personalized content for every use case with just a text script, Brett Lindenberg, CEO of the interactive vide company Mindstamp, told PYMNTS. "This saves brands hundreds of hours and tens of thousands of dollars compared to creating a traditional video that requires planning, equipment and personnel," he added. "Tools like Colossyan and Synthesia allow companies to skip directly to the final product and continually revise it to meet emerging needs." While AI's role in content creation is growing, McDonald believes human creativity will remain crucial. "AI will understand your customers best, but only you can bridge the gap between your clients and yourself," he says, advising businesses to "stay true to your color and use AI as a tool to help that color brighten."

Meta Introduces MovieGen a New Text To Video AI Model

Meta has unveiled MovieGen, a groundbreaking AI model that transforms text prompts into high-quality videos, marking a significant leap in artificial intelligence capabilities. This innovative system has quickly captured the attention of the AI community, outperforming existing state-of-the-art models and even specialized video generation companies. "Movie Gen sets a new standard for immersive AI content. Our latest research breakthroughs demonstrate how you can use simple text inputs to produce custom videos and sounds, edit existing videos or transform your personal image into a unique video." Meta At the heart of MovieGen lies a sophisticated model boasting 30 billion parameters, allowing the creation of intricate and lifelike video content. This core model works in tandem with MovieGen Audio, a 13 billion parameter system designed to produce high-quality audio clips that seamlessly integrate with the generated video. The personalized extensions, MovieGen Video and MovieGen Edit, allow users to fine-tune and customize their video content after the initial training phase, offering unprecedented flexibility in AI-driven video creation. MovieGen sets itself apart with its remarkable ability to maintain fidelity to lighting, physics, and prompt adherence. The system excels in managing complex visual elements, including: This attention to detail ensures that the resulting videos are not only visually compelling but also maintain a high degree of realism, making them suitable for a wide range of applications. Here are a selection of other articles from our extensive library of content you may find of interest on the subject of AI video : MovieGen's sophisticated editing features set it apart from other AI video generation tools. The system supports text-based video editing, allowing users to add effects or change backgrounds with remarkable precision. Its ability to accurately simulate physics and adapt to changes in edited videos underscores its potential as a powerful video editing tool. These advanced editing capabilities could transform professional video editing software like Adobe Premiere Pro and DaVinci Resolve. By allowing personalized video creation from single images, MovieGen opens new avenues for filmmakers and content creators, potentially saving significant time and resources in the production process. MovieGen's audio generation capabilities are equally impressive. The system produces high-quality audio tracks that match video content with remarkable accuracy, thanks to its extensive training dataset. This ensures: The seamless integration of audio and video generation in a single system represents a significant advancement in AI-driven content creation. MovieGen's impact on the content creation landscape is substantial. By streamlining video and audio production processes, it provides filmmakers and content creators with a powerful tool to innovate and produce media content more efficiently. The system's capabilities have the potential to: As MovieGen continues to evolve, it promises to reshape the filmmaking and media production industries, ushering in a new era of AI-driven content creation. The system's ability to generate high-quality video and audio content from simple text prompts represents a significant step forward in the field of artificial intelligence and its applications in creative industries. While MovieGen's capabilities are impressive, it's important to note that the technology is still evolving. As with any AI tool, ethical considerations and potential impacts on creative industries will need to be carefully monitored and addressed as the technology becomes more widely adopted. To learn more about the new AI video generator released by Meta jump over to the official website.

Meta showcases 'Movie Gen' to create short video from text prompts - no training, skills, or fancy equipment required

Forward-looking: Advances in AI are attempting to remove many of the prohibitive financial and technical challenges that can make content creation more challenging outside of big cloud data centers. AI research teams are developing and refining tools allowing even the most novice creators to generate content based on nothing more than a description of what they want to see. This week, Meta published a blog post highlighting Movie Gen, the company's upcoming text-to-content audio and video generator. Movie Gen was introduced this week by Mark Zuckerberg in an Instagram post reminding us that we shouldn't skip leg day. The preview comes several months after OpenAI's introduction of its Sora AI text-to-video solution. A longer and more informative announcement was posted on Meta's AI research team blog, providing additional information about Movie Gen's capabilities. The new tool will support text-based video and audio generation and editing capabilities. The post includes several videos demonstrating Movie Gen's ability to create and modify content easily based on a user's text inputs, producing video content up to 16 seconds in length at a rate of 16 frames per second. The tool can also generate highly personalized videos using a single photo in combination with a descriptive text prompt. In addition to video editing, Meta's AI solution can also create ambient sound, background music, and sound effects to accompany generated or pre-created videos. Much like its video generation capabilities, Movie Gen's audio generation is driven by user-defined text inputs. The post showcases several examples of its audio capabilities, including an ATV riding through sand dunes, a skateboarder rolling through a skate park, and penguins swimming through an AI-generated body of water filled with large floating spheres. Despite Zuckerberg's post claiming the new feature could be available for Instagram users next year, a Threads post by Meta's Chief Product Officer, Chris Cox, clarified the current status of Movie Gen's current state. Cox explained that Meta is incorporating feedback from filmmakers and video creators to prioritize features like enhanced editing capabilities and the ability to generate videos using a specific character or image. While the preview appears to show that Movie Gen already offers these features, Cox mentioned that the team isn't ready to release them as a product just yet. According to him, "...it's still expensive, and generation time is too long - but we wanted to share where we are since the results are getting quite impressive."

Meta advances generative AI video creation with Movie Gen - SiliconANGLE

Meta Platforms Inc.'s artificial intelligence research team has showcased a new family of generative AI models for media that can generate and edit videos from simple text prompts. Though the models are still a work in progress, the company said they will provide the foundation of new video creation features set to appear in Facebook, Instagram and WhatsApp next year. The Meta Movie Gen models will enable users to create high-quality HD videos and images, edit those creations, generate audio and soundtracks, and even embed their own likeness within them, the company said. In a blog post, Meta's AI team explained that it's aiming to usher in a new era of AI-generated content for creators on its platforms. The Meta Movie Gen models build on the company's earlier work in generative AI content creation, which began with its "Make-A-Scene" models that debuted in 2022, enabling users to create simple images and audio tracks, and later videos and 3D animations. Meta's later Llama Image foundation models expanded on this work, introducing higher-quality images and videos, as well as editing capabilities. "Movie Gen is our third wave, combining all of these modalities and enabling further fine-grained control for the people who use the models in a way that's never before been possible," Meta's AI team said in a blog post. According to Meta, the Movie Gen collection is made up of four models that enable video generation, personalized video generation, precise video editing and audio generation. The first of the models, Video Generation, is a 30 billion-parameter transformer model that's able to generate videos of up to 16 seconds in duration at 16 frames per second from prompts that can be simple text, images or a combination of the two. Meta explained that it's built on a joint model architecture that's optimized for both text-to-image and text-to-video, and features advanced capabilities such as the ability to reason about object motion, subject-object interactions and camera motion, so it can replicate more realistic motion in the videos it produces. The Personalized Videos model is a bit different, as it's specifically designed to take an image of the user and create videos starring them, based on the user's text prompts. Meta explained that the same foundational transformer model was used as the basis of its Precise Video Editing model. To use it, the user simply uploads the video they want to edit, along with a text input that describes how they want it to be edited, and the model will do the rest. It's all about enabling more precision for creators, who can use it to add, remove or swap out specific elements of a video, such as the background, objects in the video, or style modifications, the company said. It does this while preserving the original video content, targeting only the relevant pixels. As for the Audio Generation tool, this is based on a 13 billion-parameter audio generation model that can take both video and text inputs to create high-fidelity soundtracks of up to 45 seconds. It's able to generate ambient sound, sound effects and instrumental background music, Meta said, and synchronize this with the content in the video. Meta hasn't said anything about if or when it might make the Meta Movie Gen models available for others to use, but the company generally opts to open-source its AI innovations, such as its Llama models. So it's likely that it won't be long until developers will be able to start experimenting with them. When they do launch, Meta Movie Gen will go head-to-head with a number of other video generation models, such as Runway AI Inc.'s Gen-3 Alpha Turbo, OpenAI's upcoming Sora, Google DeepMind's Veo, Adobe Inc.'s Firefly, Luma AI Inc.'s Dream Machine and Captions LLC's video editing tools. The company is confident it can compete with those rivals. It separately published a research paper for those who want a more exhaustive deep dive into the inner workings of the Meta Movie Gen models. In the paper, it claims a number of breakthroughs in model architecture, training objectives, data recipes, inference optimizations and evaluation protocols, and it believes these innovations enable Meta Movie Gen to significantly outperform its competitors. That said, Meta concedes that there's still a lot of room for improvement in its models, and it's planning on making further optimizations to decrease inference time and improve the quality of the videos it generates. Holger Mueller of Constellation Research Inc. said generative AI has already revolutionized the way people write text, create images, understand documents and fix code, and the industry is now turning to the harder task of video creation. "Creating film and video is a slow and expensive process that costs lots of money," Mueller said. "Meta is promising to give creators a faster and much more affordable alternative with Meta Movie Gen, and it could potentially democratize movie creation. If it does, it'll likely send a few shockwaves across the traditional movie industry." Meta said the next steps involve working closely with filmmakers and other creators to integrate their feedback into the Meta Movie Gen models, with the goal being to come up with a finished product that's ultimately destined to appear on platforms like Facebook and Instagram. "Imagine animating a 'day in the life' video to share on Reels and editing it using text prompts, or creating a customized animated birthday greeting for a friend and sending it to them on WhatsApp," the company said. "With creativity and self-expression taking charge, the possibilities are infinite."

Meta Announces Movie Gen, an AI Video and Audio Creator

Meta has created a generative AI video tool that will allow its users to create and edit video as well as audio based on text prompts. The feature, called Movie Gen, can even create video footage from an uploaded photo of someone. Movie Gen won't be available to Meta's large number of users on platforms such as Facebook and Instagram until at least next year. Currently, it's been tested internally and by a small number of filmmakers, the company said. Meta shared a research paper for Movie Gen in which the company says that its model will produce high-quality 1080p HD videos with different aspect ratios and synchronized audio. The video feature is Meta's latest announcement in the AI space. Facebook users can already generate AI stickers and the company has said it wants to roll out more AI chatbot features on platforms like Instagram. The company is even adding AI features to its Ray Ban smart glasses. But as it moves into AI-generated video, Meta will be competing with companies such as OpenAI and Google, two tech giants that have already debuted AI video models. The company touted Movie Gen's features with a series of videos and stills on its blog post, including Perzonalized Video, which will take a still photo of a person and create full-motion videos based on that image. It will also use text prompts to edit AI videos on the fly; in an example the company used, props such as pom-poms could be added to a video with a text request. In addition to its video features, Movie Gen will also be able to create sound effects and full soundtracks, Meta said.

Meta's Movie Gen AI Video Generator Is Capable of Making Actual Movies, Music Included

Meta's Movie Gen AI video generator can take images of people and turn them into videos, edit existing videos, and even create music and sound effects. Meta’s AI journey would inevitably take it into the budding realm of AI video. Now, the Mark Zuckerberg-led company has Movie Gen, another video generator capable of making some realistic-ish video from a short text prompt. Meta claims this is as useful for Hollywood as it is for the average Instagrammer. Movie Gen can create audio, making it the most capable deep fake generator we’ve seen yet. In a blog post, Meta showed off a few example videos, including a happy baby hippo swimming underwater, somehow floating just below the surface and apparently having no problems holding its breath. Other videos showcase penguins dressed in “Victorian†outfits with too-short sleeves and skirts to be representative of the time period and a woman DJing next to a cheetah who is too distracted by the beat to care about her present danger. Everybody’s getting in on the AI-generated video space. Already this year, Microsoft’s VASA-1 and OpenAI’s Sora promised “realistic†videos generated from simple text prompts. Despite being teased back in February, Sora has yet to see the light of day. Meta’s Movie Gen offers a few more capabilities than the competition, including editing existing video with a text prompt, creating video based on an image, and adding AI-generated sound to the created video. The video editing suite seems especially novel. It works on generated video as well as real-world captures. Meta claims its model “preserves the original content†while adding elements to the footage, whether they’re backdrops or outfits for the scene’s main characters. Meta showed how you can also take pictures of people and drop them into generated movies. Meta already has music and sound generation models, but the social media giant displayed a few examples of the 13B parameter audio generator adding sound effects and soundtracks on top of videos. The text input could be as simple as “rustling leaves and snapping twigs†to add to the generated video of a snake winding along the forest floor. The audio generator is currently limited to 45 seconds, so it won’t score entire movies. At least, it won’t be just yet. And no, sorry, you can’t use it yet. Meta’s chief product officer, Chris Cox, wrote on Threads, “We aren’t ready to release this as a product anytime soonâ€"it’s still expensive, and generation time is too long.†In its whitepaper discussing Movie Gen, Meta said the whole software suite is made up of multiple foundation models. The largest video model the company has is a 30B parameter transformer model with a maximum context length of 73,000 video tokens. The audio generator is a 13B parameter foundation model that can do both video-to-audio and text-to-audio. It’s hard to compare that to the biggest AI companies' video generators, especially since OpenAI claims Sora uses “data called patches, each of which is akin to a token in GPT.†Meta is one of the few major companies that still release data with its new AI tools, a practice that has fallen by the wayside as AI has become excessively commercialized. Despite that, Meta’s whitepaper doesn’t offer much of an idea of where it got its training data for Movie Gen. In all likelihood, some part of the data set has come from Facebook users’ videos. Meta also uses the photos you take with the Meta Ray-Ban smart glasses to train its AI models. You can’t use Movie Gen yet. Instead, other AI movie generators like RunwayML’s Gen 3 offer a limited number of tokens to create small clips before you need to start paying. A report by 404 Media earlier this year indicated that Runway trained its AI from thousands of YouTube videos, and like most AI startups, it never asked permission before scraping that content. Meta said it worked closely with filmmakers and video producers when creating this model and will continue doing so as it works on Movie Gen. Reports from earlier this year indicate studios are already cozying up to AI companies. Independent darling A24 has recently worked with VC firms specializing in AI, with some tied to OpenAI. On the flip side, Meta is reportedly in talks with Hollywood stars like Judi Dench and Awkwafina about using their voices for future AI projects.

Meta unveils Movie Gen text-to-video and sound AI model

Meta on Friday introduced Movie Gen, an AI-powered text-to-video and sound generator designed to create and edit videos based on text inputs. Movie Gen allows users to transform photos into videos and generate or extend soundtracks based on prompts. This launch positions Meta's tool alongside leading media generation platforms, including OpenAI's Sora, which has yet to be publicly released. Meta aims to democratize creativity, stating that whether someone is an aspiring filmmaker or a casual content creator, everyone should have "access to tools that help enhance their creativity." According to their latest research, Movie Gen enables users to produce custom videos and sounds using simple text inputs. In comparison, tests, Movie Gen outperformed other models in the industry. This tool is part of Meta's ongoing commitment to sharing AI research with the public. Meta's journey began with the "Make-A-Scene" series, which allowed users to create images, audio, video, and 3D animations. With diffusion models, Meta advanced to the Llama Image foundation models, enabling higher-quality image and video generation. Movie Gen represents the third phase of this development, merging multiple modalities to provide users with more control than ever before. Meta stresses that while generative AI offers exciting applications, it is not a replacement for artists and animators. Instead, Movie Gen aims to empower users to express themselves creatively and produce high-definition videos and audio. 1. Video Generation: Movie Gen uses a 30B parameter transformer model to generate videos up to 16 seconds long at 16 frames per second. It integrates both text-to-image and text-to-video techniques, handling object motion, subject interactions, and camera movements with precision. 2. Personalized Video Generation: Meta's tool can take an individual's image and create a personalized video using text prompts. This feature excels at preserving human identity and motion, according to Meta. 3. Precise Video Editing: Movie Gen allows users to edit videos with high accuracy, supporting both localized edits (e.g., adding or removing elements) and global edits (e.g., changing backgrounds or styles) without affecting the overall content. 4. Audio Generation: Meta trained a 13B parameter model to generate audio up to 45 seconds long, including sound effects, background music, and ambient sounds. All audio is synced with the video, and an audio extension feature enables coherent sound generation for longer videos. Meta's foundation models have driven technical innovations in architecture, training methods, and evaluation protocols. Human evaluators consistently preferred Movie Gen over industry alternatives across the four main capabilities. Meta has shared a detailed 92-page research paper outlining the technical insights of Movie Gen. Despite its promising potential, Meta acknowledges that Movie Gen has some limitations, including long generation times and the need for further optimization. They are actively working on improving these aspects as development continues. Meta plans to collaborate with filmmakers and creators to refine Movie Gen based on user feedback. The company envisions a future where users can create personalized videos, share content on platforms like Reels, or generate custom animations for apps like WhatsApp. Meta's chief product officer, Chris Cox, shared on Threads that Movie Gen is not ready for public release due to high costs and slow processing times, though the results are promising. Meanwhile, CEO Mark Zuckerberg announced that Movie Gen will be coming to Instagram next year, showcasing a video created by the tool.

Meta's Movie Gen Can Create Eerie Videos and Audio

The next natural step in the generative AI era is generative videos, but that's much more complex than just generating pictures or text. Meta's Movie Gen AI wants to be a comprehensive interface for all your video-generating needs, but it's still not publicly available. Meta has announced Movie Gen, a new set of generative AI models that can create and edit videos and audio using text prompts. The models are said to outperform similar models in the industry and represent a significant advancement in AI-powered media creation. Movie Gen will let you generate videos up to 16 seconds long from text prompts, as well as generate videos featuring a specific person based on their image and a text prompt. You can also edit existing videos using text prompts, adding, removing, or replacing elements with precision, and it can even create audio up to 45 seconds long that is synchronized with video content, including ambient sound, sound effects, and background music, to go with those generated videos. Video generation is something the company has been working on for a long while, and now, those efforts are finally closer to a finished product. Meta claims that Movie Gen achieves state-of-the-art results in several areas, including video quality, personalization, editing accuracy, and audio-video alignment. The company attributes these achievements to technical innovations in model architecture, training data, and evaluation methods. The company aims to eventually collaborate with filmmakers and creators to ensure that Movie Gen becomes a valuable tool for creative expression. Potential applications of Movie Gen include creating animated videos for social media, generating personalized greetings, and editing videos using simple text commands. Meta isn't giving us a real timeline for when we might expect to check this out. AI video generation, in general, is in an early stage right now and we don't have models that are as advanced as image/text models currently are. This might change that if it releases soon. Source: Meta

Meta's Movie Gen AI Model Can Generate Video With Sound or Music

Meta says that users can also provide instructions to edit videos Facebook owner Meta announced on Friday it had built a new AI model called Movie Gen that can create realistic-seeming video and audio clips in response to user prompts, claiming it can rival tools from leading media generation startups like OpenAI and ElevenLabs. Samples of Movie Gen's creations provided by Meta showed videos of animals swimming and surfing, as well as videos using people's real photos to depict them performing actions like painting on a canvas. Movie Gen also can generate background music and sound effects synced to the content of the videos, Meta said in a blog post, and use the tool to edit existing videos. In one such video, Meta had the tool insert pom-poms into the hands of a man running by himself in the desert, while in another it changed a parking lot where a man was skateboarding from dry ground into one covered by a splashing puddle. Meta's Movie Gen AI model lets users edit videos using text prompts Photo Credit: Meta Videos created by Movie Gen can be up to 16 seconds long, while audio can be up to 45 seconds long, Meta said. It shared data showing blind tests indicating that the model performs favorably compared with offerings from startups including Runway, OpenAI, ElevenLabs and Kling. The announcement comes as Hollywood has been wrestling with how to harness generative AI video technology this year, after Microsoft-backed OpenAI in February first showed off how its product Sora could create feature film-like videos in response to text prompts. Technologists in the entertainment industry are eager to use such tools to enhance and expedite filmmaking, while others worry about embracing systems that appear to have been trained on copyright works without permission. Lawmakers also have highlighted concerns about how AI-generated fakes, or deepfakes, are being used in elections around the world, including in the U.S., Pakistan, India and Indonesia. Meta spokespeople said the company was unlikely to release Movie Gen for open use by developers, as it has with its Llama series of large-language models, saying it considers the risks individually for each model. They declined to comment on Meta's assessment for Movie Gen specifically. Instead, they said, Meta was working directly with the entertainment community and other content creators on uses of Movie Gen and would incorporate it into Meta's own products sometime next year. According to the blog post and a research paper about the tool released by Meta, the company used a mix of licensed and publicly available datasets to build Movie Gen. OpenAI has been meeting with Hollywood executives and agents this year to discuss possible partnerships involving Sora, although no deals have been reported to have come out of those talks yet. Anxieties over the company's approach increased in May when actor Scarlett Johansson accused the ChatGPT maker of imitating her voice without permission for its chatbot. Lions Gate Entertainment, the company behind "The Hunger Games" and "Twilight," announced in September that it was giving AI startup Runway access to its film and television library to train an AI model. In return, it said, the studio and its filmmakers can use the model to augment their work. © Thomson Reuters 2024

Meta to Add New AI-Powered Video Generation Capabilities to Apps | PYMNTS.com

Meta has unveiled generative artificial intelligence (AI) research that shows how simple text inputs can be used to create custom videos and sounds and edit existing videos. Dubbed "Meta Movie Gen," this AI model builds upon the company's earlier generative AI models Make-A-Scene and Llama Image, the company said in a Friday (Oct. 4) blog post. Movie Gen combines the modalities of those earlier generation models and enables further fine-grained control, according to the post. Meta is currently making this new model available to only some employees and outside partners but plans to integrate it into some of its existing apps next year, Bloomberg reported Friday, citing its interview with Connor Hayes, a Meta vice president focused on generative AI products. Before making Movie Gen available to the public, Meta aims to prevent the technology from being used to make videos of people without their consent, according to the report. Meta said in an April blog post that three of its platforms would label AI-manipulated content as "Made with AI." In Meta's Friday blog post, the company said: "While there are many exciting use cases for these foundation models, it's important to note that generative AI isn't a replacement for the work of artists and animators. We're sharing this research because we believe in the power of this technology to help people express themselves in new ways and to provide opportunities to people who might not otherwise have them." Movie Gen's video generation capabilities enable it to generate 16-second videos with 16 frames per second, while accounting for object motion, subject-object interactions and camera motion, according to the post. The foundation model can also generate personalized videos based on a person's image and a text prompt, the post said. With its video editing capabilities, Movie Gen can add, remove or replace elements and modify the background or style, per the post. It targets only the relevant pixels, leaving the original content otherwise preserved. The model can also generate up to 45 seconds of audio based on a video and text prompts, adding ambient sound, sound effects and background music that are synced to the video content, according to the post. Potential future uses of this technology include editing videos to share on Reels and creating animated birthday greetings to send via WhatsApp, per the post. "As we continue to improve our models and move toward a potential future release, we'll work closely with filmmakers and creators to integrate their feedback," the post said. "By taking a collaborative approach, we want to ensure we're creating tools that help people enhance their inherent creativity in new ways they may have never dreamed would be possible."

Meta Movie Gen just unveiled -- new AI video generator rivals OpenAI's Sora

Meta's Movie Gen can generate footage from text, turn images into videos and even add music and sound effects Meta announced a new AI-powered video and audio generator today (October 4) to rival the likes of OpenAI's Sora and ElevenLabs' AI voice tools. Called Movie Gen, it can create realistic-looking video and audio clips based on user prompts as well as edit existing footage or images with text. While there's no timeline yet on when Movie Gen will be available to the public, samples of its creations shared by Meta showed videos of thunder cracking overhead in the desert, animals swimming, and a child running along the beach with a kite generated by text prompts. The software suite can also generate audio like ambient noise, sound effects, and background music that syncs to the imagery, Meta said in a blog post. Movie Gen can create 1080p videos up to 16 seconds long in different aspect ratios, while audio can be up to 45 seconds long, Meta said. In addition to generating new clips, Movie Gen can be used to edit existing video footage as well. In one such video, Meta had the tool replace a floating lantern with a string of bubbles. In another, it inserted pom-poms into the hands of a man running in the desert, while a third replaced his clothes with an inflatable dinosaur costume, showing how further tweaks can be made via text prompts. Meta also demonstrated how it can generate a video of a person using just a still image and a text prompt. Chris Cox, chief product officer at Meta Platforms, said a full release is still a ways away. "We aren't ready to release this as a product anytime soon -- it's still expensive and generation time is too long -- but we wanted to share where we are since the results are getting quite impressive," he wrote on Threads. The announcement makes Meta the latest major tech company pushing full speed ahead into the AI space. In the past few months, both Google and Adobe have unveiled AI video generators to compete with third-generation AI video tools from leading media generation startups. In its announcement, Meta shared data showing blind tests that indicate Movie Gen performs favorably compared with offerings from startups like Runway, OpenAI, ElevenLabs and Kling. So while we'll have to wait and see whether Movie Gen ranks among our list of the best AI video generators, the early results are promising.

You'll want to try Meta's amazing new AI video generator

Too bad you can't unless you're a pro filmmaker Meta asked to work with Meta has shared another contestant in the AI video race that's seemingly taken over much of the industry in recent months. The tech giant released a new model called Movie Gen, which, as the name indicates, generates movies. It's notably more comprehensive in its feature list than many others in its initial rollout, comparable to OpenAI's Sora model, which garnered so much attention upon its initial unveiling. That said, Movie Gen also shares with Sora a limitation on access to specific filmmakers partnering with Meta rather than a public rollout. Movie Gen is impressive based on the demonstrations of its ability to produce movies from text prompts, as seen above. The model can make 16-second videos and upscale them to 1080p resolution. The caveat is that the video comes out at 16 frames per second, a speed slower than any filming standard. For a more normal 24 fps, the film clip can't be more than 10 seconds long. Still, 10 seconds can be plenty with the right prompt. Meta gave Movie Gen a fun personalization feature reminiscent of its Imagine tool for making images with you in them. Movie Gen can do the same with a video, using a reference image to put a real person into a clip. If the model can regularly match the demonstration, a lot of filmmakers might be eager to try it. And the videos aren't just limited to a prompt that then has to be rewritten to make another video that you hope will be better. Movie Gen has a text-based editing feature where a prompt can narrowly adjust one bit of the film or change an aspect of it as a whole. You might ask for characters to wear different outfits or set the background to a different location. That flexibility is impressive. The adjustments extend to the camera moves too, with panning and tracking requests understood by the AI and incorporated into the video or its later edits. The awareness of objects and their movements is likely borne out of the SAM 2 model Meta recently released, which is capable of tagging and tracking objects in videos. Good visuals are all too common now among AI video makers, but Meta is going for the audio side of filmmaking too. Movie Gen will use the text prompts for the video to produce a soundtrack that blends with the sight, putting rain sounds in a rainy scene or car engines revving to go with a film set in a traffic jam. It will even create new music to play in the background and try to match the mood of the prompted video. Human speech is not currently part of Movie Gen's repertoire. Meta has kept impressive AI engines from the public before, most notably with an AI song generator it said was too good to release due to concerns over misuse. The company didn't claim that as the reason for keeping Movie Gen away from most people, but it wouldn't be surprising if it was a contributing factor. Still, going the OpenAI Sora route means Meta has to ignore the possibility of a more open rival winning some of its market share. And there are an awful lot of AI video generators out or coming soon. That includes the new or recently upgraded models from Runway, Pika, Stability AI, Hotshot, and Luma Labs' Dream Machine, among many others.

Meta's Movie Gen lets users animate photos into videos

Meta on Friday provided a look at a generative artificial intelligence model it is working on that lets people create short videos, complete with audio, from text prompts and photos. The tech titan touted "Meta Movie Gen" as a "breakthrough" when it comes to using an AI engine to crank out video and audio. Meta said it will get feedback from filmmakers and creators as it eases toward making the video-generating AI engine publicly available. No release plans for Movie Gen were disclosed. "While there are many exciting use cases for these foundation models, it's important to note that generative AI isn't a replacement for the work of artists and animators," Meta said in a post. The model lets people use text prompts to create custom videos, including clips based on people's photos, of up to 16 seconds long, according to Meta. "Our model achieves state-of-the-art results when it comes to creating personalized videos that preserve human identity and motion," Meta said. "Imagine animating a 'day in the life' video to share on Reels and editing it using text prompts or creating a customized animated birthday greeting for a friend and sending it to them on WhatsApp." Movie Gen is the third wave of AI-powered video generation at Meta, according to the company. The rise of AI-powered tools for creating realistic videos has prompted worries about "deepfakes" that pirate people's likenesses.

Meta's Movie Gen looks like a huge leap forward for AI video (but you can't use it yet)

It can craft videos with sound, and also let you easily edit any video using text. At this point, you probably either love the idea of making realistic videos with generative AI, or you think it's a morally bankrupt endeavor that devalues artists and will usher in a disastrous era of deepfakes we'll never escape from. It's hard to find middle ground. Meta isn't going to change minds with Movie Gen, its latest video creation AI model, but no matter what you think of AI media creation, it could end up being a significant milestone for the industry. Movie Gen can produce realistic videos alongside music and sound effects at 16 fps or 24 fps at up to 1080p (upscaled from 768 by 768 pixels). It can also generative personalized videos if you upload a photo, and crucially, it appears to be easy to edit videos using simple text commands. Notably, it can also edit normal, non-AI videos with text. It's easy to imagine how that could be useful for cleaning up something you've shot on your phone for Instagram. Movie Gen is just purely research at the moment -- Meta won't be releasing it to the public, so we have a bit of time to think about what it all means. The company describes Movie Gen as its "third wave" of generative AI research, following its initial media creation tools like Make-A-Scene, as well as more recent offerings using its Llama AI model. It's powered by a 30 billion parameter transformer model that can make 16 second-long 16 fps videos, or 10-second long 24 fps footage. It also has a 13 billion parameter audio model that can make 45 seconds of 48kHz of content like "ambient sound, sound effects (Foley), and instrumental background music" synchronized to video. There's no synchronized voice support yet "due to our design choices," the Movie Gen team wrote in their research paper. Meta says Movie Gen was initially trained on "a combination of licensed and publicly available datasets," including around 100 million videos, a billion images and a million hours of audio. The company's language is a bit fuzzy when it comes to sourcing -- Meta has already admitted to training its AI models on data from every Australian user's account, it's even less clear what the company is using outside of its own products. As for the actual videos, Movie Gen certainly looks impressive at first glance. Meta says that in its own A/B testing, people have generally preferred its results compared to OpenAI's Sora and Runway's Gen3 model. Movie Gen's AI humans look surprisingly realistic, without many of the gross telltale signs of AI video (disturbing eyes and fingers, in particular). "While there are many exciting use cases for these foundation models, it's important to note that generative AI isn't a replacement for the work of artists and animators," the Movie Gen team wrote in a blog post. "We're sharing this research because we believe in the power of this technology to help people express themselves in new ways and to provide opportunities to people who might not otherwise have them." It's still unclear what mainstream users will do with generative AI video, though. Are we going to fill our feeds with AI video, instead of taking our own photos and videos? Or will Movie Gen be deconstructed into individual tools that can help sharpen our own content? We can already easily remove objects from the backgrounds of photos on smartphones and computers, more sophisticated AI video editing seems like the next logical step.

Meta Shows Off its 'Industry-Leading' AI Video Generator Called Movie Gen

Meta has previewed its AI video generator called Movie Gen by showing some impressive-looking examples, but there's no word on a release date. The AI video space is hotting up while simultaneously slowing down, it has now been seven months since OpenAI announced the much-hyped Sora, and yet we are still none the wiser as to when it will be released. Small AI video startups have been filling the gap releasing interesting models but it is still up in the air as to who will be the most successful player in the burgeoning AI video market. Part of the problem is AI video models are simply not good products yet. Users have to wait a long time for videos to appear after typing in a prompt, something acknowledged by Meta's chief product officer Chris Cox. "We aren't ready to release this as a product anytime soon -- it's still expensive and generation time is too long -- but we wanted to share where we are since the results are getting quite impressive," Cox writes on Threads. Nevertheless, Cox says the model is "industry-leading". Videos on Movie Gen can be generated via a text prompt but also with a still image or moving footage. The model also has the capability to match AI-generated audio with AI imagery and videos can be made in different aspect ratios -- features that already exist on competitors' models. One example Meta gave on the Movie Gen page is footage of a runner in the desert. It was tweaked so he had blue pom poms in one version; so that he was running through a cactus desert in another; and in another version his running clothes were replaced with an inflatable dinosaur costume. Meta, like all AI companies, was vague about what data Movie Gen was trained on. Saying only it was "a combination of licensed and publicly available datasets" in a blog post. In July it was revealed that AI video startup Runway scraped hundreds of YouTube video to make its latest Gen-3 model. It is believed that Meta has an advantage over the competition thanks to its insistence that it has the right to use the data created by its millions of social media users for the purposes of AI training.

Meta Unveils Movie Gen; its state-of-the art Video Editor

Meta AI launched the Movie Gen, a 13B parameter model designed for video and text to audio generation - one of the most advanced media foundation models to-date. Its chief features include generating videos from text, editing videos with text, producing personalised videos, and creating sound effects. While the model is not publicly available yet, the demos and use cases are already taking over the internet. . "We're continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future," read the statement hinting on its potential release, hopefully next year. To this end, Meta's Chief, Mark Zuckerberg, shared a video on Instagram of himself doing leg press, created by Movie Gen. He teased that the feature would come to Instagram by 2025. True to its open-research roots, Meta AI released a technical paper, spanning 90 pages, explaining the methodology behind the foundational model. It is important to note that Meta hasn't mentioned whether this model would be open-source yet. The capabilities are at par, or some suggest even better than Sora, which is yet to be released. OpenAI's flagship model is not publicly available yet, nor is there any update from the company regarding its release - eight months since its announcement. The timing of this is interesting, as Tim Brooks, a member of the Sora team, announced his departure from OpenAI today to join its rival, Google DeepMind. Needless to say, the video generation space is growing increasingly competitive. Last month, an NYC based AI video startup, Runway, partnered with Lionsgate to introduce AI into filmmaking. Runway's goal was to provide cutting-edge tools for artists to bring their stories to life, and with this deal, possibly to the big screen too.

Meta's Movie Gen Brings AI-Powered Video, Audio Creation To All - Meta Platforms (NASDAQ:META)

Meta's AI advancements aim to boost creator capabilities and strengthen user engagement. Meta Platforms Inc. META launched its latest innovation in generative AI Friday with the release of Movie Gen, an advanced tool for creating high-quality videos, images, and audio through simple text prompts. What Happened: Building on its established AI models, Movie Gen promises to revolutionize content creation by offering capabilities in video generation, editing, and audio production, the company said. This announcement continues Meta's trend of pushing the boundaries of AI technologies for creators worldwide. Also Read: Mark Zuckerberg Bulldozes Past Jeff Bezos To Become World's 2nd Richest Person As Meta Stock Soars: Is Musk's Top Spot In Jeopardy? Meta's latest AI research, Movie Gen, introduces a suite of models that enable high-quality video and audio production using only text prompts. The AI system offers multiple functionalities, from personalized video generation to precise video editing and audio creation. According to Meta, these tools outperform competitors when evaluated by human users. By combining these innovations, Meta hopes to empower content creators with unprecedented creative capabilities. Why It Matters: Meta's AI breakthroughs continue to play a pivotal role in the company's broader strategy of driving user engagement and product adoption. According to analysts, these AI and mixed reality advances further strengthen Meta's position in the tech industry and could drive significant revenue growth. The company's recent Meta Connect event showcased hardware innovations, such as the Quest 3 headset and Orion glasses. Movie Gen's introduction highlights Meta's continued focus on cutting-edge AI research. As AI models become more integrated with products like the Quest 3 headset and Orion glasses, Meta's diverse technological ecosystem is expected to grow, contributing to a strong competitive position. Read Next: If You Invested $1,000 In Facebook Stock When Mark Zuckerberg Changed Name To Meta, Here's How Much You'd Have Today Image created using artificial intelligence via Midjourney. This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Market News and Data brought to you by Benzinga APIs

Who needs Sora when you've got Meta Movie Gen? | Digital Trends

Meta revealed Movie Gen, its third-wave multimodal video AI, on Friday. It promises to "produce custom videos and sounds, edit existing videos, and transform your personal image into a unique video," while outperforming similar models like Runway's Gen-3, Kuaishou Technology's Kling 1.5, or OpenAI's Sora. Meta Movie Gen builds off of the company's earlier work, first with its multimodal Make-A-Scene models, and then Llama's image foundation models. Movie Gen is a collection of all of these models -- specifically, video generation, personalized video generation, precise video editing, and audio generation -- that improves the creator's fine-grain control. "We anticipate these models enabling various new products that could accelerate creativity," the company wrote in its announcement post. Recommended Videos For video generation, Movie Gen relies on a 30B-parameter model that outputs up to 16-second clips, albeit at a pokey 16 frames per second (fps). "These models can reason about object motion, subject-object interactions, and camera motion, and they can learn plausible motions for a wide variety of concepts," Meta said, "making them state-of-the-art models in their category." Using that same model, Movie Gen can create personalized videos for creators from still images. Meta employs a variant of that video-generation model that uses both video- and text-based inputs to precisely edit the content that it generates. It can affect both localized edits such as adding, removing, or replacing elements, and global edits like applying a new cinematic style. To generate audio, Movie Gen relies on a separate 13B-parameter model that can create up to 45 seconds of audio -- be it ambient background noise, sound effects, or instrumental scores -- while automatically syncing that content to the video. According to Meta's white paper, Movie Gen consistently won out in A/B tests against other state-of-the-art video AIs including Gen3, Sora, and Kling 1.5 in the category of video generation. It also topped ID-animator in personalized video generation and Pika Labs Sound Gen for audio generation. It also bested Gen3 a second time, in video editing capabilities. Based on the demo videos we've seen so far, Movie Gen far outclasses the current batch of free-to-use video generators as well. The company says it plans to "work closely with filmmakers and creators to integrate their feedback" as it continues to develop these models, but was quick to point out that it has no intention of displacing human creators with AI. " We're sharing this research because we believe in the power of this technology to help people express themselves in new ways and to provide opportunities to people who might not otherwise have them," the company wrote. "Our hope is that perhaps one day in the future, everyone will have the opportunity to bring their artistic visions to life and create high-definition videos and audio using Movie Gen."

Meta unveils AI video models, as tech giants race to produce Hollywood-style clips

Meta unveils Movie Gen -- its most advanced effort to create video clips from text prompts A baby hippo swims in a lagoon. A fire dancer performs on a beach as waves crash into the shore. A DJ plays tunes on a turntable. Those are Hollywood-style movie clips produced by Meta's new AI models, which can transform text to video from a simple prompt. The technology, called Movie Gen, uses two new foundational AI models the company highlighted in a research paper Friday that enable users to make custom movies, edit videos, and create a video using a person's image. "We really wanted to improve the state of the technology," said Ahmad Al-Dahle, vice president of Meta's generative AI division. "They [are] exceeding our expectations on quality in terms of natural motion or aesthetics." Meta is not yet releasing Movie Gen to users, or providing the models open-source to developers, but it is making the evaluation data on the underlying prompts available to researchers to review. The race to create AI models that can generate video is heating up. In February, OpenAI unveiled its own text-to-video model called Sora, which the company says can produce eye-popping videos that are up to a minute long. In May, YouTube debuted Veo, a video generation model, and Imagen 3, new text-to-image model that the company says will "support the creative process." Movie Gen follows similar cinematographic efforts from Meta including Make-A-Video and Emu Video, which generate less advanced videos from text. Meta's new model is expected to power future products that could enable creators to share more AI-generated content on its social media networks. The company has pushed out new AI tools in recent months to allow users to create their own chatbots, alter silly images of themselves, and birth imaginary scenes from text prompts. But some critics have argued that easy AI video tools could open the floodgates to misinformation online and hurt the job market in Hollywood. While some of the biggest studios have long relied on artificial intelligence to help create sophisticated cinematic effects, the latest advances in generative artificial intelligence have provoked fears among actors that the technology could eliminate jobs. Al-Dahle said video generation tools will include a watermark that identifies them as generated using artificial intelligence. Meta said Movie Gen can produce high-quality HD videos of up to 16 seconds from a text prompt and can create videos featuring a specific person, as depicted in a photograph. The model also allows people to edit existing videos with simple text prompts, such as adding thundercrack sounds or tinsel streamers to a lantern, according to the research paper. "They have the ability to do what we call very precise editing," Al-Dahle said. "So if you're asking to edit, for example, a tree and remove it, we only remove the tree."

How Meta Movie Gen could usher in a new AI-enabled era for content creators

Whether a person is an aspiring filmmaker hoping to make it in Hollywood or a creator who enjoys making videos for their audience, we believe everyone should have access to tools that help enhance their creativity. Today, we're excited to premiere Meta Movie Gen, our breakthrough generative AI research for media, which includes modalities like image, video, and audio. Our latest research demonstrates how you can use simple text inputs to produce custom videos and sounds, edit existing videos, and transform your personal image into a unique video. Movie Gen outperforms similar models in the industry across these tasks when evaluated by humans. This work is part of our long and proven track record of sharing fundamental AI research with the community. Our first wave of generative AI work started with the Make-A-Scene series of models that enabled the creation of image, audio, video, and 3D animation. With the advent of diffusion models, we had a second wave of work with Llama Image foundation models, which enabled higher quality generation of images and video, as well as image editing. Movie Gen is our third wave, combining all of these modalities and enabling further fine-grained control for the people who use the models in a way that's never before been possible. Similar to previous generations, we anticipate these models enabling various new products that could accelerate creativity. While there are many exciting use cases for these foundation models, it's important to note that generative AI isn't a replacement for the work of artists and animators. We're sharing this research because we believe in the power of this technology to help people express themselves in new ways and to provide opportunities to people who might not otherwise have them. Our hope is that perhaps one day in the future, everyone will have the opportunity to bring their artistic visions to life and create high-definition videos and audio using Movie Gen.

Meta Says Its New Movie Gen AI Is an Industry First -- But a Demo Shows It Isn't Perfect

In a demonstration to the New York Times, Movie Gen made a mistake while trying to create a video of a dog talking into a cellphone. Meta previewed new AI tools on Friday called Movie Gen that can create videos, edit them automatically, and layer on AI-generated sound for a cohesive video clip. Movie Gen works with written text prompts, an image, or an existing video as input. There's also an option to add a personal picture so users can see themselves in the video. Related: Meta Is Putting AI Images on Your Facebook and Instagram Feeds, With Personalized Pictures After the AI works its magic and generates a video, a user can type in a text prompt to create a custom audio soundtrack to play with the video. While Meta says Movie Gen's high-definition videos are "the first of its kind in the industry," when creating long videos at different aspect ratios, it doesn't mean the AI is perfect. Right now the AI can only generate videos that last up to 16 seconds -- and it doesn't always get the assignment right. In a demonstration to the New York Times, Meta's AI tool made a mistake. Though it was able to create a video of a dog in a park talking into a phone, the AI messed up by placing a human hand around the phone instead of a dog's paw. Chris Cox, chief product officer at Meta, stated in a Threads post that Movie Gen is "industry-leading" in video quality but that Meta isn't prepared to release the tools because they're too expensive and the videos currently take too long to generate. Meta is sharing what it has right now because the outputs "are getting quite impressive," Cox wrote. Meta isn't the first to show off a text-to-video AI generator tool -- ChatGPT-maker OpenAI did it in February with its text-to-video model Sora. In July, OpenAI published multiple YouTube videos in partnership with artists and entrepreneurs showing how Sora could create fantastical short films.

Meta's AI-Powered Video Generator is Scary Impressive - Phandroid

With the tech industry pushing AI-based software all over, AI video generators at this point aren't anything new, although admittedly a lot of them are a bit lacking. This is something that Meta apparently wants to address with its new AI video tool, dubbed "Movie Gen." As with most gen AI software, Movie Gen works by creating content based on user prompts, and it can even edit existing footage by adding in visual elements based on what you type, allowing users to add objects into videos which weren't there before. It can also transform still photos into videos - the software's webpage demonstrates several sample videos, showing subject photos which are then used in realistic-looking footage. Additionally, it can also create sound effects, ambient noise and cinematic soundtracks for use with footage, again all with text-based prompts. Of course all of this brings into question how malicious actors can use such technology and software for all the wrong reasons, as well as creatives concerned about the impact that AI has on the film and music industry, although Meta states that it's developing its AI software "around a core set of values to ensure trust and safety."

Meta's Movie Gen Makes Convincing AI Video Clips

An AI-generated video made from the prompt "Baby Hippo Lagoon."Courtesy of Meta Meta just announced its own media-focused AI model, called Movie Gen, that can be used to generate realistic video and audioclips. The company shared multiple 10-second clips generated with Movie Gen, including a Moo Deng-esque baby hippo swimming around, to demonstrate its capabilities. While the tool is not yet available for use, this Movie Gen announcement comes shortly after its Meta Connect event, which showcased new and refreshed hardware and the latest version of its large language model, Llama 3.2. Going beyond the generation of straightforward text-to-video clips, the Movie Gen model can make targeted edits to an existing clip, like adding an object into someone's hands or changing the appearance of a surface. In one of the example videos from Meta, a woman wearing a VR headset was transformed to look like she was wearing steampunk binoculars. Audio bites can be generated alongside the videos with Movie Gen. In the sample clips, an AI man stands near a waterfall with audible splashes and the hopeful sounds of a symphony; the engine of a sports car purrs and tires screech as it zips around the track, and a snake slides along the jungle floor, accompanied by suspenseful horns. Meta shared some further details about Movie Gen in a research paper released Friday. Movie Gen Video consists of 30 billion parameters, while Movie Gen Audio consists of 13 billion parameters. (A model's parameter count roughly corresponds to how capable it is; by contrast, the largest variant of Llama 3.1 has 405 billion parameters.) Movie Gen can produce high-definition videos up to 16 seconds long, and Meta claims that it outperforms competitive models in overall video quality. Earlier this year, CEO Mark Zuckerberg demonstrated Meta AI's Imagine Me feature, where users can upload a photo of themselves and role-play their face into multiple scenarios, by posting an AI image of himself drowning in gold chains on Threads. A video version of a similar feature is possible with the Movie Gen model -- think of it as a kind of ElfYourself on steroids. What information has Movie Gen been trained on? The specifics aren't clear in Meta's announcement post: "We've trained these models on a combination of licensed and publicly available data sets." The sources of training data and what's fair to scrape from the web remain a contentious issue for generative AI tools, and it's rarely ever public knowledge what text, video, or audioclips were used to create any of the major models. It will be interesting to see how long it takes Meta to make Movie Gen broadly available. The announcement blog vaguely gestures at a "potential future release." For comparison, OpenAI announced its AI video model, called Sora, earlier this year and has not yet made it available to the public or shared any upcoming release date (though WIRED did receive a few exclusive Sora clips from the company for an investigation into bias). Considering Meta's legacy as a social media company, it's possible that tools powered by Movie Gen will start popping up, eventually, inside of Facebook, Instagram, and WhatsApp. In September, competitor Google shared plans to make aspects of its Veo video model available to creators inside its YouTube Shorts sometime next year. While larger tech companies are still holding off on fully releasing video models to the public, you are able to experiment with AI video tools right now from smaller, upcoming startups, like Runway and Pika. Give Pikaffects a whirl if you've ever been curious what it would be like to see yourself cartoonishly crushed with a hydraulic press or suddenly melt in a puddle.

Meta unveils advanced video-creation AI

Why it matters: Video-creation models have emerged as a major focus for a number of tech giants, including Meta, OpenAI and Adobe -- but Movie Gen, like many rival products, is a ways off from general public availability. Driving the news: Meta unveiled Movie Gen and detailed its capabilities in a 90-page research paper. What they're saying: "This is the most advanced video generation system that we know of," Ahmad Al-Dahle, vice president of generative AI at Meta, told Axios. The big picture: Video creation is one of the widely eyed uses for generative AI, for everything from entertainment to corporate messaging. Startups like Runway have focused on the creative end, with HeyGen and others focused on enterprise use cases. Yes, but: Many of the tools, including OpenAI's Sora, are not yet available. OpenAI officials earlier this week declined to offer an update on when it might be made available more broadly. Between the lines: Broadly releasing video tools like these raises safety concerns, and providing these services at scale requires a vast amount of processing power.

Meta AI turns pictures into videos with sound

The tech titan touted "Meta Movie Gen" as a "breakthrough" when it comes to using an AI engine to crank out video and audio. Meta said it will get feedback from filmmakers and creators as it eases toward making the video-generating AI engine publicly available. No release plans for Movie Gen were disclosed. "While there are many exciting use cases for these foundation models, it's important to note that generative AI isn't a replacement for the work of artists and animators," Meta said in a post. The model lets people use text prompts to create custom videos, including clips based on people's photos, of up to 16 seconds long, according to Meta. "Our model achieves state-of-the-art results when it comes to creating personalized videos that preserve human identity and motion," Meta said. "Imagine animating a 'day in the life' video to share on Reels and editing it using text prompts or creating a customized animated birthday greeting for a friend and sending it to them on WhatsApp." Movie Gen is the third wave of AI-powered video generation at Meta, according to the company. The rise of AI-powered tools for creating realistic videos has prompted worries about "deepfakes" that pirate people's likenesses.

No one can actually use Meta's newest AI tool, Movie Gen

Meta just announced a new generative AI model to help users turn simple text into images, videos, and audio clips. Meta Movie Gen allows users to input text and automatically generate new videos, personalize videos, do video editing, and generate audio, all of which is trained on a "combination of licensed and publicly available datasets." If that seems a lot like OpenAI's Sora, that's because it is quite similar. But Movie Gen builds upon Meta's previous work with AI -- you might remember, or even use, Llama Image, for instance. But Movie Gen is the third wave of its generative AI work, which Meta says combines all of its previous modalities. "Similar to previous generations," Meta wrote in a blog post, "we anticipate these models enabling various new products that could accelerate creativity." Meta notes that "generative AI isn't a replacement for the work of artists and animators," but says the company "believe[s] in the power of this technology to help people express themselves in new ways and to provide opportunities to people who might not otherwise have them." "Our hope is that perhaps one day in the future, everyone will have the opportunity to bring their artistic visions to life and create high-definition videos and audio using Movie Gen," the blog post reads. The entire post about this new technology is focused on the future, noting that you can't actually use Movie Gen yet. Chris Cox, Meta's chief product officer, wrote on Threads that the company isn't "ready to release this as a product anytime soon -- it's still expensive and generation time is too long -- but we wanted to share where we are since the results are getting quite impressive." The company said it was looking to tweak the model in the lead-up to it hitting the public. "As we continue to improve our models and move toward a potential future release, we'll work closely with filmmakers and creators to integrate their feedback," Meta wrote in the blog post. "By taking a collaborative approach, we want to ensure we're creating tools that help people enhance their inherent creativity in new ways they may have never dreamed would be possible. Imagine animating a 'day in the life' video to share on Reels and editing it using text prompts, or creating a customized animated birthday greeting for a friend and sending it to them on WhatsApp. With creativity and self-expression taking charge, the possibilities are infinite."

Meta announces new AI model that can generate video with sound

Facebook owner claims Movie Gen can create realistic-seeming video and audio clips that rival competitors' Meta, the owner of Facebook and Instagram, announced on Friday it had built a new artificial intelligence model called Movie Gen that can create realistic-seeming video and audio clips in response to user prompts, claiming it can rival tools from leading media generation startups like OpenAI and ElevenLabs. Samples of Movie Gen's creations provided by Meta showed videos of animals swimming and surfing, as well as clips using people's real photos to depict them performing actions like painting on a canvas. Movie Gen also can generate background music and sound effects synced to the content of the videos, Meta said in a blog post. Users can also edit existing videos with the model. In one such video, Meta had the tool insert pom-poms into the hands of a man running by himself in the desert, while in another it changed a parking lot on which a man was skateboarding from dry ground into one covered by a splashing puddle. Videos created by Movie Gen can be up to 16 seconds long, while audio can be up to 45 seconds long, Meta said. It shared data showing blind tests indicating that the model performs favorably compared with offerings from startups including Runway, OpenAI, ElevenLabs and Kling. The announcement comes as Hollywood has been wrestling with how to harness generative AI video technology this year, after Microsoft-backed OpenAI in February first showed off how its product Sora could create feature film-like videos in response to text prompts. Technologists in the entertainment industry are eager to use such tools to enhance and expedite filmmaking, while others worry about embracing systems that appear to have been trained on copyright works without permission. Lawmakers also have highlighted concerns about how AI-generated fakes, or deepfakes, are being used in elections around the world, including in the US, Pakistan, India and Indonesia. Meta spokespeople said the company was unlikely to release Movie Gen for open use by developers, as it has with its Llama series of large-language models, saying it considers the risks individually for each model. They declined to comment on Meta's assessment for Movie Gen specifically. Instead, they said, Meta was working directly with the entertainment community and other content creators on uses of Movie Gen and would incorporate it into Meta's own products sometime next year. According to the blog post and a research paper about the tool released by Meta, the company used a mix of licensed and publicly available datasets to build Movie Gen. OpenAI has been meeting with Hollywood executives and agents this year to discuss possible partnerships involving Sora, although no deals have been reported to have come out of those talks yet. Anxieties over the company's approach increased in May when the actor Scarlett Johansson accused the ChatGPT maker of imitating her voice without permission for its chatbot. Lions Gate Entertainment, the company behind The Hunger Games and Twilight, announced in September that it was giving the AI startup Runway access to its film and television library to train an AI model. In return, it said, the studio and its film-makers can use the model to augment their work.

Meta unveils AI models that create photorealistic movies with sound

Meta debuted a suite of artificial intelligence models called "Movie Gen" on Oct. 4 capable of generating photorealistic movies up to 16-seconds long complete with sound effects and backing music tracks. Movie Gen isn't the first multimodal AI model capable of generating video and audio from simple text prompts, but it appears to demonstrate state-of-the-art capabilities. The researchers responsible for the application's development claim it outperformed rival systems in human testing. Movie Gen According to a blog post from Meta, Movie Gen is currently capable of outputting movies up to 16-seconds long at a frame rate of 16 frames per second (FPS). To put this into perspective, Hollywood films prior to the digital age were traditionally shot at 24 FPS to achieve what's called the "film look." While higher FPS rates are considered better in gaming and other graphical applications, Meta's 16 FPS isn't far off from what would be considered professional quality movie imagery. The Movie Gen models can generate completely novel movies based on simple text prompts or modify existing images or videos to replace or modify objects and backgrounds. Its most advanced contribution, however, may be the AI suite's ability to generate up to 45-seconds of audio featuring sound effects and background music. According to Meta, Movie Gen integrates and syncs the audio to the motion in the generated videos. Related: Meta shows off Web3-to-reality bridge with 'Hyperscape' metaverse demo Research only Meta is keeping the foundation models behind Movie Gen under wraps for the time being. The company hasn't given a timeframe for the product's launch and says it will require further safety testing before deployment. Per a research paper from Meta's AI team: "The Movie Gen cast of foundation models were developed for research purposes and need multiple improvements before deploying them ... when we do deploy these models, we will incorporate safety models that can reject input prompts or generations that violate our policies to prevent misuse."

Meta, challenging OpenAI, announces new AI model that can generate video with sound

Meta CEO Mark Zuckerberg holds a smartphone, as he makes a keynote speech at the Meta Connect annual event, at the company's headquarters in Menlo Park, California, U.S. September 25, 2024. Facebook owner Meta announced on Friday it had built a new AI model called Movie Gen that can create realistic-seeming video and audio clips in response to user prompts, claiming it can rival tools from leading media generation startups like OpenAI and ElevenLabs. Samples of Movie Gen's creations provided by Meta showed videos of animals swimming and surfing, as well as videos using people's real photos to depict them performing actions like painting on a canvas. Movie Gen also can generate background music and sound effects synced to the content of the videos, Meta said in a blog post, and use the tool to edit existing videos. In one such video, Meta had the tool insert pom-poms into the hands of a man running by himself in the desert, while in another it changed a parking lot where a man was skateboarding from dry ground into one covered by a splashing puddle. Videos created by Movie Gen can be up to 16 seconds long, while audio can be up to 45 seconds long, Meta said. It shared data showing blind tests indicating that the model performs favorably compared with offerings from startups including Runway, OpenAI, ElevenLabs and Kling. The announcement comes as Hollywood has been wrestling with how to harness generative AI video technology this year, after Microsoft-backed OpenAI in February first showed off how its product Sora could create feature film-like videos in response to text prompts. Technologists in the entertainment industry are eager to use such tools to enhance and expedite filmmaking, while others worry about embracing systems that appear to have been trained on copyright works without permission. Lawmakers also have highlighted concerns about how AI-generated fakes, or deepfakes, are being used in elections around the world, including in the U.S., Pakistan, India and Indonesia. Meta spokespeople said the company was unlikely to release Movie Gen for open use by developers, as it has with its Llama series of large-language models, saying it considers the risks individually for each model. They declined to comment on Meta's assessment for Movie Gen specifically. Instead, they said, Meta was working directly with the entertainment community and other content creators on uses of Movie Gen and would incorporate it into Meta's own products sometime next year. According to the blog post and a research paper about the tool released by Meta, the company used a mix of licensed and publicly available datasets to build Movie Gen. OpenAI has been meeting with Hollywood executives and agents this year to discuss possible partnerships involving Sora, although no deals have been reported to have come out of those talks yet. Anxieties over the company's approach increased in May when actor Scarlett Johansson accused the ChatGPT maker of imitating her voice without permission for its chatbot. Lions Gate Entertainment, the company behind "The Hunger Games" and "Twilight," announced in September that it was giving AI startup Runway access to its film and television library to train an AI model. In return, it said, the studio and its filmmakers can use the model to augment their work.

Meta AI turns pictures into videos with sound

SAN FRANCISCO (AFP) - Meta on Friday provided a look at a generative artificial intelligence model it is working on that lets people create short videos, complete with audio, from text prompts and photos. The tech titan touted "Meta Movie Gen" as a "breakthrough" when it comes to using an AI engine to crank out video and audio. Meta said it will get feedback from filmmakers and creators as it eases toward making the video-generating AI engine publicly available. No release plans for Movie Gen were disclosed. "While there are many exciting use cases for these foundation models, it's important to note that generative AI isn't a replacement for the work of artists and animators," Meta said in a post. The model lets people use text prompts to create custom videos, including clips based on people's photos, of up to 16 seconds long, according to Meta. "Our model achieves state-of-the-art results when it comes to creating personalized videos that preserve human identity and motion," Meta said. "Imagine animating a 'day in the life' video to share on Reels and editing it using text prompts or creating a customized animated birthday greeting for a friend and sending it to them on WhatsApp." Movie Gen is the third wave of AI-powered video generation at Meta, according to the company. The rise of AI-powered tools for creating realistic videos has prompted worries about "deepfakes" that pirate people's likenesses.

I regret mocking Meta's AI video efforts - Movie Gen looks scarily impressive

I may have been a little quick to mock two years ago when Meta announced that it was working on an AI video generator with the stunningly unimaginative name of Make-A-Video. Coming on the heels of CEO Mark Zuckerberg's misguided Metaverse obsession, it seemed like another silly way for the company that owns Facebook and Instagram to blow vast amounts of resources researching something that had little to do with its core business and which produced creepy, glitchy video that could barely serve for a meme. That was in October 2022. Make-A-Video never saw a public release, but Meta continued tinkering away. Now it's just provided a glimpse of its latest AI video generator. And while Movie Gen might have only a slightly better name, its output looks light years ahead of its predecessor. It might even be better than the likes of Runway and Open AI's Sora. Judging from Meta's Movie Gen research paper, the company's new AI video model can create impressively coherent video that closely adheres to text prompts. But there's more: text prompts can also be used to generate audio in the video, matching music and sound effects with the imagery. The model can also edit existing videos, and still images can be used as references. Meta says that Movie Gen can create videos of up to 16 seconds in different aspect ratios and audio of up to 45 seconds. It claims that the model compares well against Runway, OpenAI's Sora and Kling, and the few examples provided back that up. Human and animal anatomy, clothing and other objects are generally convincing if still a little fuzzy or overly sharpened and cartoonlike at times. Lighting and shadows look particularly impressive. And a public release? While OpenAI initially said that Sora would see a public release this year, Meta seems to be in no hurry to release its model. On threads, chief product officer Chris Cox wrote "We aren't ready to release this as a product anytime soon -- it's still expensive and generation time is too long -- but we wanted to share where we are since the results are getting quite impressive." Sora and Runway have been regarded as the most powerful AI video generators so far, and the developer of the latter has recently entered into a deal to create a bespoke model for the film and TV producer Lionsgate. If it gets a release, Meta's AI video generator is also likely to compete against Adobe Firefly Video, which the company behind Photoshop and Premiere Pro has promised is "coming soon". For the moment, Meta's model looks more powerful, but Firefly Video is likely to have practical advantages like direct use in Premiere Pro. It's not clear how either model was trained. There have been claims that Runway was trained on videos scrapped from YouTube, which YouTube's CEO has said would be a breach of the platforms terms of use. Meta says that Movie Gen was trained on a "a combination of licensed and publicly available datasets". It didn't specify which, but Meta has access to vast amounts of video data directly through its own products, Instagram and Facebook.

Meta's Movie Gen model puts out realistic video with sound, so we can finally have infinite Moo Deng

No one really knows what generative video models are useful for just yet, but that hasn't stopped companies like Runway, OpenAI, and Meta from pouring millions into developing them. Meta's latest is called Movie Gen, and true to its name turns text prompts into relatively realistic video with sound... but thankfully no voice just yet. And wisely they are not giving this one a public release. Movie Gen is actually a collection (or "cast" as they put it) of foundation models, the largest of which is the text-to-video bit. Meta claims it outperforms the likes of Runway's Gen3, LumaLabs' latest, and Kling1.5, though as always this type of thing is more to show that they are playing the same game than that Movie Gen wins. The technical particulars can be found in the paper Meta put out describing all the components. Audio is generated to match the contents of the video, adding for instance engine noises that correspond with car movements, or the rush of a waterfall in the background, or a crack of thunder halfway through the video when it's called for. It'll even add music if that seems relevant. It was trained on "a combination of licensed and publicly available datasets" that they called "proprietary/commercially sensitive" and would provide no further details on. We can only guess means is a lot of Instagram and Facebook videos, plus some partner stuff and a lot of others that are inadequately protected from scrapers -- AKA "publicly available." What Meta is clearly aiming for here, however, is not simply capturing the "state of the art" crown for a month or two, but a practical, soup-to-nuts approach where a solid final product can be produced from a very simple, natural-language prompt. Stuff like "imagine me as a baker making a shiny hippo cake in a thunderstorm." For instance, one sticking point for these video generators has been in how difficult they usually are to edit. If you ask for a video of someone walking across the street, then realize you want them walking right to left instead of left to right, there's a good chance the whole shot will look different when you repeat the prompt with that additional instruction. Meta is adding a simple, text-based editing method where you can simply say "change the background to a busy intersection" or "change her clothes to a red dress" and it will attempt to make that change, but only that change. Camera movements are also generally understood, with things like "tracking shot" and "pan left" taken into account when generating the video. This is still pretty clumsy compared with real camera control, but it's a lot better than nothing. The limitations of the model are a little weird. It generates video 768 pixels wide, a dimension familiar to most from the famous but outdated 1024×768, but which is also three times 256, making it play well with other HD formats. The Movie Gen system upscales this to 1080p, which is the source of the claim that it generates that resolution. Not really true, but we'll give them a pass because upscaling is surprisingly effective. Weirdly, it generates up to 16 seconds of video... at 16 frames per second, a frame rate no one in history has ever wanted or asked for. You can, however, also do 10 seconds of video at 24 FPS. Lead with that one! As for why it doesn't do voice... well, there are likely two reasons. First, it's super hard. Generating speech is easy now, but matching it to lip movements, and those lips to face movements, is a much more complicated proposition. I don't blame them for leaving this one til later, since it would be a minute-one failure case. Someone could say "generate a clown delivering the Gettysburg Address while riding a tiny bike in circles" -- nightmare fuel primed to go viral. The second reason is likely political: putting out what amounts to a deepfake generator a month before a major election is... not the best for optics. Crimping its capabilities a bit so that, should malicious actors try to use it, it would require some real work on their part, is a practical preventive step. One certainly could combine this generative model with a speech generator and an open lip syncing one, but you can't just have it generate a candidate making wild claims. "Movie Gen is purely an AI research concept right now, and even at this early stage, safety is a top priority as it has been with all of our generative AI technologies," said a Meta rep in response to TechCrunch's questions. Unlike, say, the Llama large language models, Movie Gen won't be publicly available. You can replicate its techniques somewhat by following the research paper, but the code won't be published, except for the "underlying evaluation prompt dataset," which is to say the record of what prompts were used to generate the test videos.

Meta Unveils Instant A.I. Video Generator That Adds Sounds

In February, the artificial intelligence start-up OpenAI unveiled technology called Sora that let people generate photorealistic videos -- like woolly mammoths trotting through a snowy meadow -- simply by typing a sentence into a box on a computer screen. Because of concerns over how the technology might be misused, and perhaps the high cost of operating the technology, OpenAI has not yet released Sora beyond a small group of testers. But others companies are racing to release similar technology. On Friday, the tech giant Meta unveiled a set of A.I. tools, called Meta Movie Gen, for automatically generating videos, instantly editing them and synchronizing them with A.I.-generated sounds effects, ambient noise and background music. "Video is nothing without audio," said Ahmad Al-Dahle, Meta's vice president of generative A.I., said in an interview. Given a short text description of an image, one tool creates a video. Then, given another description of some sounds, a second tool adds the audio. A demonstration included short videos -- created in minutes -- of a man in a poncho standing over a waterfall, a snake slithering through a forest and a person riding an all-terrain vehicle across the desert. Each included music as well as sound effects. The new system also let people upload photos of themselves and instantly weave these images to moving videos. It generates 16-frame-per-second videos that last for up to 16 seconds. In some cases, these videos are flawed. During one demonstration for The New York Times, when asked to generate a video of a dog in a park talking on a cellphone, it mistakenly grafted a human hand onto the phone. Meta, the owner of Facebook and Instagram, sees the technology as a way to accelerate the work of Hollywood moviemakers and online creators. Like OpenAI, it has started testing the technology with a small group of professionals. (The Times sued OpenAI and its partner, Microsoft, in December, claiming copyright infringement of news content related to A.I. systems.) The two companies are among many developing this kind of tool; others include start-ups like Runway and Pika and tech giants like Google and Microsoft. Though many believe the technology could speed the work of seasoned moviemakers, it could also replace less experienced digital artists. Others experts worry that the technology could become a quick and inexpensive way of creating online disinformation, making it even harder to tell what's real on the internet. While OpenAI and other companies have been reluctant to release some A.I. technologies amid concerns about disinformation and other potential risks, Meta has been quicker to share them freely, arguing that the risks are not as great as they may seem. Mr. Al-Dahle said Meta would tag videos produced by the system with watermarks that identified them as being generated by A.I. But such tags can be removed. (The Times added "Generated by A.I." watermarks to the videos with this article.) He also said the company was still testing the technology to better understand what the risks might be and how they could be mitigated. Meta's demonstration did not include spoken words. But many companies, including OpenAI, are developing A.I. technologies that can instantly recreate human voices.

Meta enters AI video wars with powerful Movie Gen set to hit Instagram in 2025

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Meta founder and CEO Mark Zuckerberg, who built the company atop of its hit social network Facebook, finished this week strong, posting a video of himself doing a leg press exercise on a machine at the gym on his personal Instagram (a social network Facebook acquired in 2012). Except, in the video, the leg press machine transforms into a neon cyberpunk version, an Ancient Roman version, and a gold flaming version as well. As it turned out, Zuck was doing more than just exercising: he was using the video to announce Movie Gen, Meta's new family of generative multimodal AI models that can make both video and audio from text prompts, and allow users to customize their own videos, adding special effects, props, costumes and changing select elements simply through text guidance, as Zuck did in his video. The models appear to be extremely powerful, allowing users to change only selected elements of a video clip rather than "re-roll" or regenerate the entire thing, similar to Pika's spot editing on older models, yet with longer clip generation and sound built in. Meta's tests, outlined in a technical paper on the model family released today, show that it outperforms the leading rivals in the space including Runway Gen 3, Luma Dream Machine, OpenAI Sora and Kling 1.5 on many audience ratings of different attributes such as consistency and "naturalness" of motion. Meta has positioned Movie Gen as a tool for both everyday users looking to enhance their digital storytelling as well as professional video creators and editors, even Hollywood filmmakers. Advanced multimodal media capabilities Movie Gen represents Meta's latest step forward in generative AI technology, combining video and audio capabilities within a single system. Specificially, Movie Gen consists of four models: 3. Personalized Movie Gen Video - a version of Movie Gen Video post-trained to generate personalized videos based on a person's face 4. Movie Gen Edit - a model with a novel post-training procedure for precise video editing These models enable the creation of realistic, personalized HD videos of up to 16 seconds at 16 FPS, along with 48kHz audio, and provide video editing capabilities. Designed to handle tasks ranging from personalized video creation to sophisticated video editing and high-quality audio generation, Movie Gen leverages powerful AI models to enhance users' creative options. Key features of the Movie Gen suite include: * Video Generation: With Movie Gen, users can produce high-definition (HD) videos by simply entering text prompts. These videos can be rendered at 1080p resolution, up to 16 seconds long, and are supported by a 30 billion-parameter transformer model. The AI's ability to manage detailed prompts allows it to handle various aspects of video creation, including camera motion, object interactions, and environmental physics. * Personalized Videos: Movie Gen offers an exciting personalized video feature, where users can upload an image of themselves or others to be featured within AI-generated videos. The model can adapt to various prompts while maintaining the identity of the individual, making it useful for customized content creation. * Precise Video Editing: The Movie Gen suite also includes advanced video editing capabilities that allow users to modify specific elements within a video. This model can alter localized aspects, like objects or colors, as well as global changes, such as background swaps, all based on simple text instructions. * Audio Generation: In addition to video capabilities, Movie Gen also incorporates a 13 billion-parameter audio generation model. This feature enables the generation of sound effects, ambient music, and synchronized audio that aligns seamlessly with visual content. Users can create Foley sounds (sound effects amplifying yet solidifying real life noises like fabric ruffling and footsteps echoing), instrumental music, and other audio elements up to 45 seconds long. Meta posted an example video with Foley sounds below (turn sound up to hear it): Trained on billions of videos online Movie Gen is the latest advancement in Meta's ongoing AI research efforts. To train the models, Meta says it relied upon "internet scale image, video, and audio data," specifically, 100 million videos and 1 billion images from which it "learns about the visual world by 'watching' videos," according to the technical paper. However, Meta did not specify if the data was licensed in the paper or public domain, or if it simply scraped it as many other AI model makers have -- leading to criticism from artists and video creators such as YouTuber Marques Brownlee (MKBHD) -- and, in the case of AI video model provider Runway, a class-action copyright infringement suit by creators (still moving through the courts). As such, one can expect Meta to face immediate criticism for its data sources. The legal and ethical questions about the training aside, Meta is clearly positioning the Movie Gen creation process as novel, using a combination of typical diffusion model training (used commonly in video and audio AI) alongside large language model (LLM) training and a new technique called "Flow Matching," the latter of which relies on modeling changes in a dataset's distribution over time. At each step, the model learns to predict the velocity at which samples should "move" toward the target distribution. Flow Matching differs from standard diffusion-based models in key ways: * Zero Terminal Signal-to-Noise Ratio (SNR): Unlike conventional diffusion models, which require specific noise schedules to maintain a zero terminal SNR, Flow Matching inherently ensures zero terminal SNR without additional adjustments. This provides robustness against the choice of noise schedules, contributing to more consistent and higher-quality video outputs . * Efficiency in Training and Inference: Flow Matching is found to be more efficient both in terms of training and inference compared to diffusion models. It offers flexibility in terms of the type of noise schedules used and shows improved performance across a range of model sizes. This approach has also demonstrated better alignment with human evaluation results. The Movie Gen system's training process focuses on maximizing flexibility and quality for both video and audio generation. It relies on two main models, each with extensive training and fine-tuning procedures: * Movie Gen Video Model: This model has 30 billion parameters and starts with basic text-to-image generation. It then progresses to text-to-video, producing videos up to 16 seconds long in HD quality. The training process involves a large dataset of videos and images, allowing the model to understand complex visual concepts like motion, interactions, and camera dynamics. To enhance the model's capabilities, they fine-tuned it on a curated set of high-quality videos with text captions, which improved the realism and precision of its outputs. The team further expanded the model's flexibility by training it to handle personalized content and editing commands. * Movie Gen Audio Model: With 13 billion parameters, this model generates high-quality audio that syncs with visual elements in the video. The training set included over a million hours of audio, which allowed the model to pick up on both physical and psychological connections between sound and visuals. They enhanced this model through supervised fine-tuning, using selected high-quality audio and text pairs. This process helped it generate realistic ambient sounds, synced sound effects, and mood-aligned background music for different video scenes. It follows earlier projects like Make-A-Scene and the Llama Image models, which focused on high-quality image and animation generation. This release marks the third major milestone in Meta's generative AI journey and underscores the company's commitment to pushing the boundaries of media creation tools. Launching on Insta in 2025 Set to debut on Instagram in 2025, Movie Gen is poised to make advanced video creation more accessible to the platform's wide range of users. While the models are currently in a research phase, Meta has expressed optimism that Movie Gen will empower users to produce compelling content with ease. As the product continues to develop, Meta intends to collaborate with creators and filmmakers to refine Movie Gen's features and ensure it meets user needs. Meta's long-term vision for Movie Gen reflects a broader goal of democratizing access to sophisticated video editing tools. While the suite offers considerable potential, Meta acknowledges that generative AI tools like Movie Gen are meant to enhance, not replace, the work of professional artists and animators. As Meta prepares to bring Movie Gen to market, the company remains focused on refining the technology and addressing any existing limitations. It plans further optimizations aimed at improving inference time and scaling up the model's capabilities. Meta has also hinted at potential future applications, such as creating customized animated greetings or short films entirely driven by user input. The release of Movie Gen could signal a new era for content creation on Meta's platforms, with Instagram users among the first to experience this innovative tool. As the technology evolves, Movie Gen could become a vital part of Meta's ecosystem and that of creators -- pro and indie alike.

Meta Unveils 'Movie Gen' AI Video Generation Model to Challenge OpenAI Sora and Veo

Meta also demonstrates precise AI video editing. Users can remove, add, or replace elements from videos seamlessly. After the release of Llama 3.2 multimodal models, Meta has announced a state-of-the-art video generation model called Movie Gen. Meta says the foundation model is not just limited to AI video generation, but can also produce images, audio, and even edit videos. In that way, Movie Gen is a frontier media foundation model. Bear in mind that Meta has not released the model or weights, but unveiled a paper showcasing Movie Gen's capabilities. First, the Movie Gen Video model is a 30B parameter model that can generate high-definition (HD) videos of up to 16 seconds with a simple text prompt. Meta says this model can also generate high-quality images. In the demos, the generated videos look very impressive, much better than the AI videos we have seen from Runway, Pika, or Luma. In fact, it looks as good as OpenAI's Sora and Google's Veo models. Next, the Movie Gen Audio model is trained on 13B parameters and it's uniquely powerful. You can directly feed a video into this model, and Movie Gen Audio generates high-fidelity music of up to 45 seconds synced to the video. Not just that, you can add your own prompt along with the video in case you want a particular kind of sound. This model can generate ambient sound, instrumental music, and foley sound effects. Movie Gen also brings precise AI video editing. Meta says users can upload an existing or AI-generated video and it can perform targeted edits. Just like AI image editing, you can add, remove, or replace elements in videos using simple text prompts. Apart from that, users can also make broader changes like changing the background or adjusting the style. Finally, the Personalized Videos feature lets you upload your own photo and Movie Gen can create a video while preserving the original character. It also promises natural movement in videos. Overall, it seems Meta has developed a frontier media model that tightly integrates video, audio, and images.

Meta debuts AI filmmaker in challenge to OpenAI's Sora

Meta is showcasing artificial intelligence models that can generate realistic videos from text instructions, which will compete with rival offerings for filmmakers and content creators from OpenAI and Runway. Movie Gen is a suite of storytelling models that can be used for a range of tasks, such as generating videos up to 16 seconds long, video editing, matching sounds to videos and personalising video with specific images. The Instagram and Facebook owner plans to offer video-generation tools to the Hollywood filmmakers, artists and influencers who make content on its social networks. OpenAI announced its own video-generation model, Sora, in February, and has been showing it off to the film industry, although it has not yet been released as a product. While Meta released some examples of videos generated by its models on Friday, it said it did not anticipate the models being integrated into its platforms for users until next year at the earliest. "Right now . . . if you were using a video editing or video-generation feature in Instagram, that would probably not meet your expectations of how fast you would want something like that to be," said Connor Hayes, vice-president of generative AI products at Meta. "But broadly speaking, you could imagine these models being really powerful for things like Reels creation, Reels editing across the family of apps, and that's the direction that we're looking at for where we can apply it." Reels is Instagram's video creation and sharing feature. The video-generation push is part of an effort by tech companies to make tools that can be used more broadly in the entertainment industry, including advertising, as they look for ways to monetise their AI advancements. Runway, an AI video-generation start-up, signed a deal last month with entertainment company Lionsgate to train a custom model on its library of films, including Twilight and The Hunger Games. Meta claimed its videos surpassed its rivals, such as Sora and Runway, for "overall quality, motion, naturalness and consistency", citing blind human evaluations. Its models were trained on "a combination of licensed and publicly available data sets", Meta said, but would not give further details. It has used public content from its platforms, such as Facebook and Instagram, for its AI previously. The realistic nature of AI-generated videos -- and the ability to replicate people's likeness within them -- has caused concerns from workers, including actors and production staff, as to how tools may affect their jobs in future. "While there are many exciting use cases for these foundation models, it's important to note that generative AI isn't a replacement for the work of artists and animators," Meta said, emphasising that it would continue to seek feedback from filmmakers and creators. Meta said it would watermark any videos generated by the model in order to avoid copyright concerns and issues that might arise with deepfakes. "These are many of the challenges that we're going to have to work through before we can responsibly put a product out there, and that's also a big part of why this is purely a research announcement right now," Hayes added.

Meta Unveils AI Video Generator, Taking On OpenAI and Google

Facebook parent Meta Platforms Inc. debuted a new artificial intelligence tool that can generate or edit videos based on a simple text prompt, elevating competition with rivals like OpenAI and Google in the race to develop the world's most advanced AI technology. Meta's product, Movie Gen, can create a new video up to 16 seconds long based on a text prompt. It can also use such prompts to generate audio for or edit an existing video, or even use a photo to create a customized video featuring a real person.

Meta Movie Gen: Facebook's new AI Mini - Movie Tool. Here's all we know till now

Meta has announced about a new video generation and editing tool which will be totally AI driven and named it as Movie Gen. Movie Gen has some significant and striking new features which will be seen in Facebook.Movie Gen, the new video generating and editing tool was announced by Meta pretty recently which will be dominantly AI driven. According to The Hollywood Reporter, Movie Gen will be capable of creating pretty long videos nearly up to16 seconds and it can also create videos which will be solely based on the user's prompts. Movie Gen has other pretty unique features where it can be seen personalizing videos in accordance with the user by integrating uploaded photos into generated content while it will enable the users to have a of view themselves in different scenarios, the Hollywood Reporter stated. Also Read: Keeping up with her hectic schedule, here's what billionaire singer Taylor Swift eats in a day Meta's huge announcement about Movie Gen took place at an event held in Austin which is in Texas while Meta also revealed new features for Facebook platform's video which will be including a new full screen video tab focusing on reels. The Hollywood Reporter reported that all this significant new features will make the video content far more accessible and pretty engaging for users all over the world. Meta revealed that Movie Gen is only available to selected film makers and internal users as it is still in its test phase and will be available in the social media during next year. Meta claimed that the new technology is not at all intended to replace human creativity but they want to enhance the human creativity in the videos with the help of Movie Gen. Also Read : The Boys prequel: Release window of the Vought Rising revealed | What to expect Q1.What is the new video generating and editing tool by Meta? A1. Meta launched a pretty new AI driven video generating and editing tool named Movie Gen. Q2. What will be the features of Movie Gen? A2. Movie Gen will have significant features like it will be able to personalize videos by integrating uploaded photos into generated content and many more.

Meta, challenging OpenAI, announces new AI model that can generate video with sound

NEW YORK, Oct 4 (Reuters) - Facebook owner Meta (META.O), opens new tab announced on Friday it had built a new AI model called Movie Gen that can create realistic-seeming video and audio clips in response to user prompts, claiming it can rival tools from leading media generation startups like OpenAI and ElevenLabs. Samples of Movie Gen's creations provided by Meta showed videos of animals swimming and surfing, as well as videos using people's real photos to depict them performing actions like painting on a canvas. Advertisement · Scroll to continue Movie Gen also can generate background music and sound effects synced to the content of the videos, Meta said in a blog post. People can use the tool to edit existing videos as well, the company said. In one sample video, Meta had the tool insert pom-poms into the hands of a man running by himself in the desert, while in another it transformed a parking lot in a video of a man skateboarding from dry ground into one covered by a splashing puddle. Reporting by Katie Paul in New York; Additional reporting by Dawn Chmielewski in Los Angeles; Editing by Mark Porter Our Standards: The Thomson Reuters Trust Principles., opens new tab

Hippo Hype: Meta Teases 'Movie Gen' 16-Second AI Video Generator

Meta is showing off a new AI video generator that could rival OpenAI's Sora or Google's Veo. Meta says Movie Gen can create high-quality (1080p) 16-second videos with sound from a text prompt, though it did not provide a release timeline. Sora promises 60-second silent clips, while Google touts "over 60-second" clips with Veo but has not specified the audio capabilities. Neither video generator is available yet, though a Veo-powered video background generator is coming to YouTube Shorts later this year. Movie Gen's sound capabilities come from a separate AI model, dubbed Movie Gen Audio. It can "follow a text prompt" and create "high-quality cinematic sound effects and music synchronized with the video output," according to a Meta research paper. Movie Gen is targeting professional and amateur filmmakers, and says the model can help them with "multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation." Meta trained the model on "internet scale image, video, and audio data," and the demo video features a pygmy hippo, a reference to the social media craze surrounding Moo Deng, a hippo recently born in Thailand. Impersonating Moo Deng is a clever way to reveal another core capability. Movie Gen can make videos of anyone (human or hippo) from a static photo. Content creators can theoretically crank out AI-generated videos of themselves with a simple text prompt. Without proper guardrails, such as restricting video generation of public figures and politicians, this technology could lead to the rapid spread of hyper-realistic misinformation. Meta did not mention any restrictions. The announcement seems focused on assuring filmmakers that "generative AI isn't a replacement for the work of artists and animators" and insisting that it's working "closely with filmmakers and creators to integrate their feedback." OpenAI says it is working on guardrails for Sora ahead of a potential release later this year. In February, the company promised to take "several important safety steps [like] working with red teamers -- domain experts in areas like misinformation, hateful content, and bias -- who are adversarially testing the model."

Meta says its Movie Gen marks 'real' advance in AI video generation

How fake or how real is the growing stream of artificial intelligence (AI)-produced video? Turns out, there's a quantitative measure for that -- or, almost. Humans still need to decide, based on their human perception, if a video is good or not. Also: New Meta Ray-Ban AI features roll out, making the smart glasses even more tempting Mark Zuckerberg, owner of Meta Platforms, announced on Friday a new AI model called Movie Gen that can generate HD videos (1080p resolution) from a text prompt. The firm says these videos are more "realistic" on average than videos generated by competing technology (such as OpenAI's Sora text-to-video model). It can also generate synced audio, tailor the video to show a person's face, and then edit the video automatically with just a text prompt, such as, "dress the penguins in Victorian outfits" to cloak on-screen penguins. Also: OpenAI unveils text-to-video model and the results are astonishing. See for yourself In the accompanying paper, "Movie Gen: A Cast of Media Foundation Models," Meta AI researchers describe how they had humans rate the realism of the AI-generated videos: Realness: This measures which of the videos being compared most closely resembles a real video. For fantastical prompts that are out of the training set distribution (e.g., depicting fantasy creatures or surreal scenes), we define realness as mimicking a clip from a movie following a realistic art-style. We additionally ask the evaluators to select a reason behind their choice i.e., "subject appearance being more realistic" or "motion being more realistic". There is also a companion blog post. The human tests identify a win/loss score for Movie Gen versus Sora and three other prominent text-to-video AI models, Runway Gen3, Lumalabs, and Kling1.5. Also: The best AI image generators of 2024 The authors note that it's not yet possible to get good comparisons in an automated fashion. Furthermore, "assessing realness and aesthetics heavily depends on human perception and preference," they write. Not just in realism but also in the matter of how good the motion is in a video, whether it skips or fumbles parts of an action, and how faithful the video is to the text prompt entered, are things you just can't automate, they state. "We find that existing automated metrics struggle to provide reliable results, reinforcing the need for human evaluation." The benchmark measures the ways "humans prefer the results of our model against competing industry models," the paper relates, resulting in a "net win rate" in percentage terms. Also: These Meta Ray-Ban smart glasses are my favorite Prime Day deal so far The average win rate against Sora, they relate, is 11.62% of the time. The win rate against the others is substantially higher. "These significant net wins demonstrate Movie Gen Video's ability to simulate the real world with generated videos that respect physics, with motion that is both reasonable in magnitude but consistent and without distortion." They offer some sample screen grabs of videos directly in contrast to Sora. As the authors see it, "OpenAI Sora can tend to generate less realistic videos (e.g., the cartoonish kangaroo in the second row) that can be missing the motion details described in the text prompt (e.g., the non-walking robot in the bottom row)." The authors built the AI model for Movie Gen from what they call a "cast of foundation models." Also: In a surprise twist, Meta is suddenly crushing Apple in the innovation battle In the training phase, images and videos from a mixture of public and licensed data sets are compressed until the model learns to efficiently reproduce pixels of the data, the authors relate. As they term it, "We encode the RGB pixel-space videos and images into a learned spatiotemporal compressed latent space using a Temporal Autoencoder (TAE), and learn to generate videos in this latent space." That video generation is then "conditioned" on text inputs to get the model to be able to produce video in alignment with the text prompts. The parts add up to a model with 30 billion parameters -- not huge by today's training standards. Also: Meta's new $299 Quest 3S is the VR headset most people should buy this holiday season A second neural net, called "Movie Gen Audio," produces high-fidelity audio -- but for sound effects and music, not for speech. That is built on an existing approach called a "diffusion transformer," with 13 billion parameters. All that takes a lot of computing horsepower: "6,144 H100 GPUs, each running at 700W TDP and with 80GB HBM3, using Meta's Grand Teton AI server platform." Generating videos is not all Movie Gen does. In a subsequent step, the authors also subject the model to additional training to create "personalized" videos, where an individual's face can be forced to show up in the movie. Also: ChatGPT is the most searched AI tool by far, but number two is surprising They also add a final component, the ability to edit the videos with just a text prompt. The problem the authors faced is that "video editing models are hindered by the scarcity of supervised video editing data," so there aren't enough examples to give the AI model to train it. To get around that, the team went back to the Movie Gen AI model and modified it in several steps. First, they use data from image editing to simulate what is involved in editing frames of video. They put that into the training of the model at the same time as the original text-to-video training so that the AI model develops a capacity to coordinate individual frame editing with multiple frames of video. In the next portion, the authors feed the model a video, a text caption, such as "a person walking down the street," and an edited video, and train the model to produce the instruction that would lead to the change from original video to edited video. In other words, they force the AI model to associate instructions with changed videos. Also: The 4 biggest challenges of AI-generated code that Gartner left out of its latest report To test the video editing capability, the authors compile a new benchmark test based on 51,000 videos collected by Meta's researchers. They also hired crowd workers to come up with editing instructions. To evaluate the editing of the videos, the Meta team asked human reviewers to rate which video was better: one created with their AI model or with the existing state-of-the-art. They also used automated measures to compare the before and after videos in the task. Also: These AI avatars now come with human-like expressions "Human raters prefer Movie Gen Edit over all baselines by a significant margin," write the authors. In all these steps, the authors break ground in coordinating the size of AI models the data, and the amount of computing used. "We find that scaling the training data, compute, and model parameters of a simple Transformer-based model trained with Flow Matching yields high-quality generative models for video or audio." However, the authors concede that the human evaluations have their pitfalls. "Defining objective criteria evaluating model generations using human evaluations remains challenging and thus human evaluations can be influenced by a number of other factors such as personal biases, backgrounds, etc." Also: Pearson launches new AI certification - with focus on practical use in the workplace The paper doesn't have any suggestions as to how to deal with those human biases. But Meta notes that they will be releasing a benchmark test for use by others, without disclosing a time frame: In order to thoroughly evaluate video generations, we propose and hope to release a benchmark, Movie Gen Video Bench, which consists of 1000 prompts that cover all the different testing aspects summarized above. Our benchmark is more than 3⇥ larger than the prompt sets used in prior work. The company also pledged to at some point offer its videos for public inspection: "To enable fair and easy comparison to Movie Gen Video for future works, we hope to publicly release our non-cherry picked generated videos for the Movie Gen Video Bench prompt set." Also: Can synthetic data solve AI's privacy concerns? This company is betting on it According to Meta, the Movie Gen model has not yet been deployed. In the conclusion of their paper, the authors write that the AI models all "need multiple improvements before deploying them." For example, the videos generated by the model "still suffer from issues, such as artifacts in generated or edited videos around complex geometry, manipulation of objects, object physics, state transformations, etc." The audio "is sometimes out of synchronization when motions are dense" such as a video of tap dancing. Despite those limitations, Movie Gen implies a path someday to a full video creation and editing suite and even tailoring a video podcast with one's own likeness.

Meta's new "Movie Gen" AI system can deepfake video from a single photo

On Friday, Meta announced a preview of Movie Gen, a new suite of AI models designed to create and manipulate video, audio, and images, including creating a realistic video from a single photo of a person. The company claims the models outperform other video-synthesis models when evaluated by humans, pushing us closer to a future where anyone can synthesize a full video of any subject on demand. The company does not yet have plans of when or how it will release these capabilities to the public, but Meta says Movie Gen is a tool that may allow people to "enhance their inherent creativity" rather than replace human artists and animators. The company envisions future applications such as easily creating and editing "day in the life" videos for social media platforms or generating personalized animated birthday greetings. Movie Gen builds on Meta's previous work in video synthesis, following 2022's Make-A-Scene video generator and the Emu image-synthesis model. Using text prompts for guidance, this latest system can generate custom videos with sounds for the first time, edit and insert changes into existing videos, and transform images of people into realistic personalized videos. Meta isn't the only game in town when it comes to AI video synthesis. Google showed off a new model called "Veo" in May, and Meta says that in human preference tests, its Movie Gen outputs beat OpenAI's Sora, Runway Gen-3, and Chinese video model Kling. Movie Gen's video-generation model can create 1080 p high-definition videos up to 16 seconds long at 16 frames per second from text descriptions or an image input. Meta claims the model can handle complex concepts like object motion, subject-object interactions, and camera movements. Even so, as we've seen with previous AI video generators, Movie Gen's ability to generate coherent scenes on a particular topic is very likely dependent on the concepts found in example videos that Meta used to train its video-synthesis model. It's worth keeping in mind that cherry-picked results from video generators often differ dramatically from typical results, and getting a coherent result may require lots of trial and error.

Lights, Camera, AI: Facebook Parent Meta Announces Its Movie Generator

Meta (META) unveiled a new artificial intelligence (AI) tool that can generate realistic videos with accompanying music from text inputs. Meta's Movie Gen can create new videos, edit existing ones, or adapt images into video form, the company said Friday. Specifically, it can create videos up to 16 seconds long that run at 16 frames per second. Movie Gen rivals the video generator from Microsoft (MSFT)-backed OpenAI, which announced its product, Sora, in February. Meta's Movie Gen isn't publicly available yet, but Meta said it's working toward a potential future release. Shares of Meta hit another record high Friday, and were up 2% at $594.25 in afternoon trading. They've gained more than two-thirds of their value since the start of the year.

Meta Unveils AI Video and Audio Generation Tools

Meta Platforms on Friday unveiled a series of artificial intelligence-powered tools for generating videos, editing videos and creating audio. The tools, called Movie Gen , are not yet publicly available, but the company plans to integrate them in its social media apps next year. A Meta spokesperson said the company is working with filmmakers and creators to understand how the tools could be

Facebook's New Pivot to Video: AI Mini-Movies

Amazon Shook Up the Streaming Ad Market in 2024. Will It Do It Again Next Year? The tech giant Meta is leaning further into video, announcing an artificial intelligence-powered video generation and editing tool, and a tweak to Facebook that will make its Reels video content more "prominent" for users of the social network. The Facebook tweaks were announced at an event in Austin, Texas Friday, while details on the the AI-generated video tool were published on the Meta AI blog. The AI tool, Meta Movie Gen, can create videos of up to 16 seconds based only on a text prompt. Perhaps even more significantly, it also allows for video editing and "personalized" video generation, where it can take a specific photo and integrate it into a video. The personalized videos could see someone upload a photo of themselves, with the tool then able to insert the person into videos. The video editing tool, meanwhile, would allow a user to take a video they shot and add objects or tweaks to it. A demo showed a penguin walking on ice in Antarctica, with the AI dressing it in a victorian-era outfit after receiving a prompt. The tool, which follows similar tools from OpenAI (Sora) and RunwayML, among others, is sure to catch the eye of Hollywood, as studios think of ways to cut costs, and some workers wonder if the tech will ultimately displace them. In the blog post, Meta sought to dissuade those concerns, writing "while there are many exciting use cases for these foundation models, it's important to note that generative AI isn't a replacement for the work of artists and animators. "We're sharing this research because we believe in the power of this technology to help people express themselves in new ways and to provide opportunities to people who might not otherwise have them," it continued. "Our hope is that perhaps one day in the future, everyone will have the opportunity to bring their artistic visions to life and create high-definition videos and audio using Movie Gen." Meanwhile, in Austin at a Facebook IRL event, the company announced a number of updated to Messenger, Marketplace, Groups, and other Facebook features. That includes AI-generated summaries of group comments, new local and explore tabs that pull together content from Reels, marketplace, groups and events, and other features. Of most interest to Hollywood, however, will be the new full-screen video tab "that gives Reels a more prominent home on Facebook." The new tab is meant to make it easier for Facebook suers to find and watch videos via Reels (Facebook's big TikTok competitor), including shortform, longform and live videos, all powered by what Facebook calls "our turbo-charged recommendations technology." "Young adults on Facebook spend almost 60% of their time on the app watching videos, and more than half of young adults watch Reels daily - so we want to make sure we're creating a video experience that helps them get valuable tips, advice and recommendations," Facebook said in a blog post announcing the changes.

Meta's AI Bootleg Moo Deng Is an Insult to Pygmy Hippos Everywhere

Meta has just announced Movie Gen, its new AI-powered video generation tool for creating realistic-looking footage -- and one-upping OpenAI's Sora, it can automatically add matching audio to its output. But notable detail is its de facto choice of mascot. Featured front and center on the product's announcement page -- and in the marketing material shared with the press -- is an AI-generated video of a cute baby hippo swimming underwater. We'd put money on that being a shameless evocation of Moo Deng, the young, plucky pygmy hippo whose adorable antics have endeared millions. It may be a cynical marketing move, but based on the instructions it was given, the AI video generational tool was spot on -- if you can overlook the AI sheen. "A baby hippo swimming in the river. Colorful flowers float at the surface, as fish swim around the hippo," reads the prompt entered into Movie Gen, as shared by Wired. "The hippo's skin is smooth and shiny, reflecting the sunlight that filters through the water." And impressively, we must admit, all of that's there. It doesn't compare to the perfect creature that is Moo Deng -- but it's all there. For new footage, Movie Gen can generate clips up to 16 seconds long at 16 frames per second, according to The New York Times. (This puts it short of the filmmaking standard of 24 fps, despite Meta claiming it can help Hollywood filmmakers in its blog post.) But it can also be used as an editing tool for existing footage, too. In a series of examples shared in the announcement, the AI is used to add pom poms to the hands of a man running across a scenic landscape, and in an even more ridiculous showing, place him in a dinosaur suit. Movie Gen can also generate audio for the footage it produces by using both video and text inputs. That includes sound effects, background music, or even entire soundtracks, Meta said. One example depicts a man gazing over a cliff as water crashes down around him, with music swelling in the background. Another shows firecrackers being shot into the sky, and the audio seems pretty well timed to the explosions, down to the crackling that follows. There's just one small thing: Movie Gen isn't available to the public yet, and it probably won't be for a while. "We aren't ready to release this as a product anytime soon -- it's still expensive and generation time is too long -- but we wanted to share where we are since the results are getting quite impressive," Chris Cox, Meta's chief product officer, wrote on Threads. It's also far from perfect. In the NYT's testing, it mistakenly grafted a human hand onto a phone that was meant to be held by a dog. To be fair, OpenAI's Sora, which generated a lot of hype when it was unveiled in February, is also yet to be made public. In the meantime, other companies like Google-backed Runway have stepped in with impressive AI video generation tools of their own -- so the race is on.