With AI tech taking the filmmaking world by storm, filmmakers are finding ways to complement, rather than automate their creative vision
In the past couple of years we have seen an unprecedented growth of AI use in audio and video post-production, with this new and exciting technology reshaping how films are made, enhanced and perceived by the audiences. Revolutionary AI tools, help filmmakers reduce production costs and save hundreds of work hours.
Many in Hollywood and beyond worry about AI’s potential harmful impact on filmmaking jobs, perceived superficiality of its output and ethnical concerns regarding the use of AI-generated performances. In this article I want to examine some positive use cases of AI, that showcase how these new developments can unlock previously untapped capabilities.
Recent releases, such as Brady Corbet’s The Brutalist and Robert Zemeckis’ Here, showcase how much AI tech has progressed in recent years in areas such as de-aging and dialogue enhancement. All the while it is saving editors countless hours of work-time and helping filmmakers stay within their budget constraints.
Has the Revival of Mid Budgets Arrived?
There has been a lot of talk in recent years regarding the death of mid budget movies. Critics and movie aficionados have long been lamenting studios choosing to finance higher-cost, higher-revenue productions, usually big franchises and sequels, over more risky, original scripts and auteur films. Low-budget productions have suffered less from this trend, due to a lower financial risk.
AI use in filmmaking can prove to be a game-changer for mid budget productions. It has already shown to help creators achieve great results, without going over the budget on expensive post production services and by saving precious (and costly) production time. Having recognized the potential of AI incorporation in post production, most forward-thinking localization studios, including ours, have significantly expanded their AI-related services in the past couple of years.
Let’s take a look at some recent examples of game-changing use of AI in post-production. These cases show how AI can help enhance filmmakers’ vision (and thus our own movie-watching experience), without fundamentally changing the final creative output.
Dialogue Enhancement in The Brutalist
The Brutalist has been at the forefront of conversations around the use of AI in moviemaking, during this year’s Oscar’s season. After it was identified that AI audio technology was used during post production, director Brady Corbet admitted that an AI tool Respeecher has been used in post. The goal was to enhance actors’ Adrien Brody’s and Felicity Jones’s Hungarian speech, by refining certain vowels and letters.
Following a predictable backlash, Corbet has defended the use of AI in filmmaking. He stated that Brody’s and Jones’s performances are “completely their own” and AI was utilized “to preserve the authenticity of Adrien and Felicity’s performances in another language..”. It was never his goal “to replace or alter them and (was) done with the utmost respect for the craft.” Additionally, the program was used only in Hungarian editing, to replace certain letters and sounds, due to actors struggling to pronounce certain difficult Hungarian words.
According to Rupal Patel, a professor of communication sciences and disorders, use of AI for Hungarian editing, can be seen as inclusive of Hungarian moviegoers, helping them maintain the feeling of immersion.
Additionally, Patel states that the use of Respeecher in the movie, is an evolution of more traditional voice tweaking techniques. Such voice editing used to be done manually, and can now be done quicker and more efficiently, saving time and efforts to sound editors. Thus, the use of AI tools in The Brutalist, helps maintain authenticity and sense of immersion, while saving multiple hours of audio post time and helping stay within budgetary constraints, due to relative affordability of modern AI tools.
De-Aging in Here
Another recent example of controversial AI use in movies, is the use of video editing tools in Robert Zemeckis’ latest work – Here. This recent production reunited Forrest Gump stars Tom Hanks and Robin Wright, in a drama about generations of couples, living in the same house, over the span of a century. More specifically, the controversy centers on the use of de-aging technology.
The movie follows the couple of Hank and Wright, over the course of decades and different stages in their lives, requiring cutting-edge de-aging technology, to convey drastic changes in appearance. Having tested multiple de-aging solutions, the VFX team settled on the AI startup Metaphysic.
According to VFX supervisor Kevin Baillie, this technology doesn’t require dots on faces or other distracting processes. Thus actors can afford to dive into their performances and focus on physicality. During the shooting, Zemeckis had two monitors on, at the same time, one showing what was being shot, and another showing the face swap in real-time, which helped him and the actors adjust the performances right on the spot.
Importantly, despite the sophisticated “digital make up” (as Zemeckis calls it), the production still utilized style & makeup and costume design crews, proving that despite advancements in technology and digital tools, traditional production teams are still essential for the filmmaking process.
Thus AI tech works to complement and enhance the work of existing crew and departments, rather than eliminating them. Additionally, the ability to see digital effects in real time and adjust performances accordingly, results in a more immersive and convincing visual experience. This is apparent to anyone who watched the impressive movie trailer. Finally, once again, the production benefits by saving multiple hours of valuable production time and costs related to it, a big boon for this $45 million mid budget production.
The Future
As one of the most revolutionary advancements in technology, AI presents its unique set of challenges and fears. Specific to filmmaking and media production, are such concerns as job displacement and deepfake technology. These concerns are valid and should be collectively considered and resolved. However, filmmakers are increasingly finding out that, when used properly, AI presents many exciting possibilities for filmmakers, actors and production crews.
Recent examples of Brady Corbet’s The Brutalist and Robert Zemeckis’s Here, showcase the way AI technology can be utilized in post production to complement creatives’ work and enhance movie-watching experience. It also brings hope for the revival of mid budget movies and original auteur cinema, categories that have been neglected by studios, due to financial risks associated with these films.
Relative affordability and efficiency of AI tools, allows for high-quality effects, editing and sound design, without going over the budget. By streamlining mundane and labor-intensive tasks, creatives can focus on storytelling and performances. The examples of The Brutalist and Here prove that collaboration between filmmakers, VFX specialists and sound designers, on one hand and AI tools on the other, can deliver visionary creative work, worthy of contending for Oscar and various other awards. Thus, there is great potential for the future, where creatives utilize AI tools to bring their artistic vision to the big screen.
