AI Reshapes Filmmaking: New Aesthetic and Democratization of Cinema

5 ways that filmmakers are using AI to create a new aesthetic

Type text into AI image and video generators, and you'll often see outputs of unusual, sometimes creepy, pictures. In a way, this is a feature, not a bug, of generative AI. And artists are wielding this aesthetic to create a new storytelling art form. The tools, such as Midjourney to generate images, Runway and Sora to produce videos, and Luma AI to create 3D objects, are relatively cheap or free to use. They allow filmmakers without access to major studio budgets or soundstages to make imaginative short films for the price of a monthly subscription. I've studied these new works as the codirector of the AI for Media & Storytelling studio at the University of Southern California.

Blurry, morphing and surreal - a new AI aesthetic is emerging in film

University of Southern California provides funding as a member of The Conversation US. Type text into AI image and video generators, and you'll often see outputs of unusual, sometimes creepy, pictures. In a way, this is a feature, not a bug, of generative AI. And artists are wielding this aesthetic to create a new storytelling art form. The tools, such as Midjourney to generate images, Runway and Sora to produce videos, and Luma AI to create 3D objects, are relatively cheap or free to use. They allow filmmakers without access to major studio budgets or soundstages to make imaginative short films for the price of a monthly subscription. I've studied these new works as the co-director of the AI for Media & Storytelling studio at the University of Southern California. Surveying the increasingly captivating output of artists from around the world, I partnered with curators Jonathan Wells and Meg Grey Wells to produce the Flux Festival, a four-day showcase of experiments in AI filmmaking, in November 2024. While this work remains dizzyingly eclectic in its stylistic diversity, I would argue that it offers traces of insight into our contemporary world. I'm reminded that in both literary and film studies, scholars believe that as cultures shift, so do the way we tell stories. With this cultural connection in mind, I see five visual trends emerging in film. 1. Morphing, blurring imagery In her "NanoFictions" series, the French artist Karoline Georges creates portraits of transformation. In one short, "The Beast," a burly man mutates from a two-legged human into a hunched, skeletal cat, before morphing into a snarling wolf. The metaphor - man is a monster - is clear. But what's more compelling is the thrilling fluidity of transformation. There's a giddy pleasure in seeing the figure's seamless evolution that speaks to a very contemporary sensibility of shapeshifting across our many digital selves. This sense of transformation continues in the use of blurry imagery that, in the hands of some artists, becomes an aesthetic feature rather than a vexing problem. Theo Lindquist's "Electronic Dance Experiment #3," for example, begins as a series of rapid-fire shots showing flashes of nude bodies in a soft smear of pastel colors that pulse and throb. Gradually it becomes clear that this strange fluidity of flesh is a dance. But the abstraction in the blur offers its own unique pleasure; the image can be felt as much as it can be seen. 2. The surreal Thousands of TikTok videos demonstrate how cringey AI images can get, but artists can wield that weirdness and craft it into something transformative. The Singaporean artist known as Niceaunties creates videos that feature older women and cats, riffing on the concept of the "auntie" from Southeast and East Asian cultures. In one recent video, the aunties let loose clouds of powerful hairspray to hold up impossible towers of hair in a sequence that grows increasingly ridiculous. Even as they're playful and poignant, the videos created by Niceaunties can pack a political punch. They comment on assumptions about gender and age, for example, while also tackling contemporary issues such as pollution. On the darker side, in a music video titled "Forest Never Sleeps," the artist known as Doopiidoo offers up hybrid octopus-women, guitar-playing rats, rooster-pigs and a wood-chopping ostrich-man. The visual chaos is a sweet match for the accompanying death metal music, with surrealism returning as a powerful form. 3. Dark tales The often-eerie vibe of so much AI-generated imagery works well for chronicling contemporary ills, a fact that several filmmakers use to unexpected effect. In "La Fenêtre," Lucas Ortiz Estefanell of the AI agency SpecialGuestX pairs diverse image sequences of people and places with a contemplative voice-over to ponder ideas of reality, privacy and the lives of artificially generated people. At the same time, he wonders about the strong desire to create these synthetic worlds. "When I first watched this video," recalls the narrator, "the meaning of the image ceased to make sense." In the music video titled "Closer," based on a song by Iceboy Violet and nueen, filmmaker Mau Morgó captures the world-weary exhaustion of Gen Z through dozens of youthful characters slumbering, often under the green glow of video screens. The snapshot of a generation that has come of age in the era of social media and now artificial intelligence, pictured here with phones clutched close to their bodies as they murmur in their sleep, feels quietly wrenching. 4. Nostalgia Sometimes filmmakers turn to AI to capture the past. Rome-based filmmaker Andrea Ciulu uses AI to reimagine 1980s East Coast hip-hop culture in "On These Streets," which depicts the city's expanse and energy through breakdancing as kids run through alleys and then spin magically up into the air. Ciulu says that he wanted to capture New York's urban milieu, all of which he experienced at a distance, from Italy, as a kid. The video thus evokes a sense of nostalgia for a mythic time and place to create a memory that is also hallucinatory. Similarly, David Slade's "Shadow Rabbit" borrows black-and-white imagery reminiscent of the 1950s to show small children discovering miniature animals crawling about on their hands. In just a few seconds, Slade depicts the enchanting imagination of children and links it to generated imagery, underscoring AI's capacities for creating fanciful worlds. 5. New times, new spaces In his video for the song "The Hardest Part" by Washed Out, filmmaker Paul Trillo creates an infinite zoom that follows a group of characters down the seemingly endless aisle of a school bus, through the high school cafeteria and out onto the highway at night. The video perfectly captures the zoominess of time and the collapse of space for someone young and in love haplessly careening through the world. The freewheeling camera also characterizes the work of Montreal-based duo Vallée Duhamel, whose music video "The Pulse Within" spins and twirls, careening up and around characters who are cut loose from the laws of gravity. In both music videos, viewers experience time and space as a dazzling, topsy-turvy vortex where the rules of traditional time and space no longer apply. Right now, in a world where algorithms increasingly shape everyday life, many works of art are beginning to reflect how intertwined we've become with computational systems. What if machines are suggesting new ways to see ourselves, as much as we're teaching them to see like humans?

How AI can help new filmmakers create movies without replacing human creativity

Cutting-edge technology and filmmaking have always gone hand in hand, with the industry often at the forefront of adopting new ways to bring stories to life on the big screen. From the integration of sound in the late 1920s to the invention of CinemaScope in the 1950s and the surge of popularity of IMAX in the 21st century, new technology has always been embraced by the film industry as a way to tell old stories in new ways. Contents How is AI currently being used across the industry?How can new filmmakers maximize AI?What can new filmmakers expect from AI in the future?Pitfalls, ethical problems, and using AI responsibly It's no different with artificial intelligence (AI), which is quickly becoming an integral part of the several stages of film production. Recent advancements in AI and other emerging technologies have proven beneficial for filmmaking, particularly in lowering the barriers to entry into the complex, and often expensive, art form. Recommended Videos How is AI currently being used across the industry? Before the rise of impressive emerging technologies following the massive popularity of tools like ChatGPT and Gemini, AI had already been part of the filmmaking process. One of the most practical applications of AI in the industry is in automating time-consuming post-production tasks, with tools like Apple's Final Cut Pro X using the technology for auto color balance and background noise removal. Adobe Premiere Pro is also widely used for its "Enhance Speech" tool, which uses AI. Several applications also offer magic masking, which makes the once-tedious process of rotoscoping a breeze thanks to AI. More recent technologies have focused specifically on where AI and filmmaking can collide. Tools like Axle AI, for instance, can be used for tasks such as face recognition, scene detection, and transcription. This allows filmmakers to quickly search through vast amounts of footage, identify specific moments, and reduce the hours spent on repetitive work. Magisto is another exciting addition to filmmakers' toolkits, with the editing software using AI to edit video footage based on emotional cues, using its Emotion Sense technology to assemble footage into polished, cohesive narratives quickly. Create Cinematic AI Videos with Runway Gen-2 Generative AI tools like Runway have also become indispensable for visual effects and editing, offering easy-to-use tools for color grading, 3D animation integration, and video editing. This AI toolkit was famously used in the movie Everything Everywhere All at Once, which needed extensive rotoscoping. In an interview with Variety, Everything Everywhere All at Once's visual effects artist Evan Halleck explained how he used the AI-powered tool Runway to speed up the process of cutting elements away from scenes shot against a green screen. He mentions how he wishes he could have started using tools like this sooner, adding that he's "calling it Hollywood 2.0 where everyone is gonna be able to make the films and the blockbusters that only a handful of people were able to before." How can new filmmakers maximize AI? For those new to the industry, particularly indie filmmakers, AI tools can greatly reduce the time and money needed to create high-quality movies. As more emerging technologies cater specifically to newcomers to the industry, filmmakers can expect lower barriers to entry, which will only encourage diversity and strengthen new voices in cinema. These tools help filmmakers focus on storytelling and creativity while AI handles much of the technical workload. Aside from the previously mentioned optimization of time-consuming tasks like editing, color grading, and organizing footage, there are so many other ways AI can impact the process of making movies. Tools like Strada AI can assist in planning and previsualization by helping budding directors stay organized by sharing footage, syncing files, and managing editing remotely, which is especially crucial for indie projects that rely on geographically dispersed teams. Making Of CIVIL WAR (2024) - Best Of Behind The Scenes & Talk With Kirsten Dunst, Alex Garland | A24 AI-powered camera systems are also transforming how filmmakers approach cinematography. Enhanced camera systems can take the guesswork out of shooting crucial scenes, with tech like DJI's AI-integrated autofocus system -- used to shoot Alex Garland's Civil War -- helping camera crews seamlessly track subjects using Lidar and AI. By leveraging these tools to shoot indie movies, filmmakers can capture high-quality shots without extensive reshoots or technical adjustments. With editing, managing data, and even camerawork being influenced by AI, new filmmakers can confidently enter the industry with a powerful set of tools that can help them succeed -- and it's only the beginning. The ongoing development of even more tools specifically geared toward the industry promises even further democratization of filmmaking, making it easier than ever for fresh talent to create movies on any budget. What can new filmmakers expect from AI in the future? A future with countless integrations of AI in cinema is within reach, especially as ETs continue to transform the filmmaking process at every stage. New filmmakers, in particular, stand to benefit from upcoming AI advancements that aim to make production elements more accessible. One of the most exciting developments is in AI's ability to understand and process video content, which tools like Twelve Labs are using for algorithms that go beyond simple scene recognition. These tools are revolutionizing media management in the production process by allowing filmmakers to search through footage based on complex criteria, such as identifying moments of tension or joy. For those dealing with extensive footage, this kind of contextual understanding can save users time spent sorting through thousands of files to identify the scenes that match specific narrative beats or moods. New and continuously improving AI tools like Luma AI also go one step beyond the more commonly used Google Earth Pro for virtual location scouting. Luma AI lets filmmakers capture any location using a smartphone and render it into a detailed, 3D virtual environment, which is particularly useful for crews working on a budget. Instead of spending on multiple site visits and on-location shoots, just one person can scout the area and use the AI-powered platform to share the model with the rest of the team. This way, directors, cinematographers, and other creative leads can assess the location remotely, plan shots, and even simulate lighting conditions within the 3D environment. As Luma.AI Co-Founder Amit Jain put it, "Now anyone can visit a real location, quickly capture a photorealistic 3D environment and share it with their production team." Luma.AI: Artsy Rugs Palo Alto In the world of 3D animation and visual effects (VFX), AI's role is expected to expand with advanced tools automating complex effects and 3D animation processes. Much like the repetitive tasks in live-action films, a huge part of animation and VFX is painstaking manual work like rendering, texturing, and more. As neural networks and deep learning techniques improve, AI tools can better help replicate realistic textures, shadows, and lighting, allowing for immersive visual effects without high production costs. These tools are already handling simpler tasks like background replacement and object removal, but future versions could streamline intricate effects such as character animation and environment rendering. For cinematographers, AI-integrated cameras could become effective team members on a busy set. These systems could potentially be advanced enough to autonomously adjust framing, focus, lighting, and camera movement based on the scene's action or emotional context. There are even more creative applications, such as the CMR M-1, which claims to be the "world's first AI-powered movie camera." As the footage is shot, it's sent off to be AI-processed via a cloud-computing service, allowing filmmakers to essentially apply customized "AI filters" to their work. While it's currently in the prototyping stage, the creators say that more powerful AI video models will allow this process to happen in real time when the camera is released. View this post on Instagram A post shared by SpecialGuestX | SGX | AI Agency (@specialguestx) Countless other potential applications promise to make complex processes in the filmmaking process more accessible to new filmmakers. By reducing costs and streamlining repetitive tasks, AI can help fresh talent break free of the traditional constraints of the industry and focus on the creative aspects that truly matter. Pitfalls, ethical problems, and using AI responsibly It's impossible to talk about all the exciting possibilities AI offers without addressing the numerous significant ethical concerns and pitfalls that come with it. As the use of AI becomes more prevalent in filmmaking, both those involved in the industry and consumers must be aware of and actively participate in the larger conversations and debates about these emerging technologies' role in cinema. The biggest and most pressing issue is the concern about job displacement not just in filmmaking but in the creative industry as a whole. As AI takes on more of the technical work, important questions are being raised about its potential impact within the industry. If AI systems can handle tasks once done by human editors, VFX artists, sound engineers, and even writers, would this mean diminishing opportunities for skilled professionals? What's more, will generative AI's ability to replicate actors' likenesses create a future where performers' voices and faces can be duplicated without their involvement? The Screen Actors Guild-American Federation of Television and Radio Artists (SAG-AFTRA) seems to think so, as this was explicitly expressed in their Hollywood strikes, which only came to a close in December 2023. Union members reached an agreement with Hollywood studios after a lengthy negotiation, with the organization warning that it likely won't be the last (they are, in fact, currently striking over the use of AI in the video game industry). New filmmakers should be mindful of these changes, as an over-reliance on AI could inadvertently contribute to the decline of certain creative professions in the industry. Another major issue is the risk of homogenized creativity, which is the result of using AI to the point of producing generic content lacking the human touch. AI systems, especially generative tools, often rely on existing datasets to produce new outputs, which can lead to repetitive or formulaic content. New filmmakers who rely too much on AI to tell their stories may find themselves mirroring popular media and rehashing old ideas, rather than breaking new ground. Recent studies have found that AI can reduce creativity and diminish critical thinking during the writing process, which would be detrimental to filmmaking that needs meaningful storytelling to truly thrive. Plagiarism and intellectual property disputes have also been highlighted in recent months, especially as the legal system catches up with the rapidly transforming landscape of AI and creative work. Several cases have been filed against generative AI platforms, with artists arguing that AI-generated content is based on large datasets that include their original creative works. If an AI tool inadvertently produces content that closely resembles another work, filmmakers could face accusations of plagiarism. For creators who may be unfamiliar with intellectual property laws, these issues could present legal challenges and put them in a position where they are unknowingly infringing on others' work. There is a myriad of other problems with AI that affect its applications in filmmaking, such as bias in algorithms and the lack of transparency in how models are trained. New filmmakers who are looking to use emerging technologies should also make an effort to understand these ethical challenges to break into the current state of the industry responsible. AI should serve as a tool to enhance, not replace, the creative process, and newcomers play an important role in shaping the changing face of cinema. With careful use, AI can be a powerful ally that can make the magic of cinema accessible to filmmakers of all backgrounds, promising a future full of innovative hits and refreshing new perspectives.