Hollywood is infamous for celebrity excess, but Tinseltown strictly controls one scandalous indulgence: swearing.
Director Scott Mann encountered these constraints after shooting the thriller Fall. Movie giant Lionsgate — best-known for the John Wick, Saw, and Hunger Games franchises — wanted to release the film in the US. But the studio had big problems. Thirty-six of them, to be precise.
“They said it had too many f*cks,” Mann tells TNW on a video call from LA.
All those f-bombs were pushing Fall towards an R rating, which would slash the potential audience. To secure the PG-13 needed to extend the reach, those profanities had to go.
Easier said than done. Reshoots would cost a bomb and post-production magic couldn’t scrub the dirty words. Thankfully, Mann had another trick up his sleeve. Quietly, the British filmmaker had been building a startup — called Flawless — that develops AI video editing tools. Fall provided a new field test: swapping f-bombs for gentler epithets.
Mann asked the cast to record cleaner verbiage. Once the audio was ready, the Flawless system went to work. The software first converted the actors’ faces into 3D models. Neural networks then analysed and reconstructed the performances. Facial expressions and lip movements were synchronised with the new dialogue.
The experiment proved successful. All 36 f-bombs were replaced without a trace. Well, nearly all of them. “I did one f*ck in the end,” Mann says. “I’m allowed one f*ck, apparently.”
Satisfied by his restraint, the ratings board gave Fall the coveted PG-13. The film became a sleeper hit, grossing a reported $21 million against a budget of just $3 million. A sequel is now shooting in Thailand.
Buoyed by his success, Mann began commercialising the software. The latest iteration is DeepEditor, an AI tool that refines dialogue and performances.
The studio system
DeepEditor can trim lines, insert pauses, or re-time delivery. It can even copy and paste performances from one shot to another. All the outputs offer Hollywood-grade 4K resolution, 16-bit colour depth, and ACES colour spaces.
Early access applications for the tool are now open. A full product release is slated for the first half of this year.
“It’s already altering where people are shooting,” says Mann. “And as it extends out, I think it’s going to completely transform how we make movies.”
It’s also not the only tool that Mann wants to transform movies. Around a decade ago, he began developing another AI system for filmmaking. Like DeepEditor, it began life on a Hollywood set.
The big break
After progressing through film school, British TV, and short films, Mann got his big Hollywood break in 2014. Lionsgate had offered him the director’s chair for the crime thriller Heist. An all-star cast led by Robert De Niro was also on board.
Mann relished the experience. “It was a complete privilege. We were very close on the movie and really happy with the English language version. But then I saw a foreign translation of the movie.”
Mann was “horrified” by the dubbing. His script had been rewritten and the actors’ gestures had mutated. The culprit, he discovered, pervaded across the industry.
The problem stemmed from Hollywood’s established translation process. When films are dubbed, the scripts are typically rewritten to fit the original mouth movements. If the new lines still don’t match the old gestures, voice actors try to synchronise the two by twisting their delivery in unnatural directions. The results range from amusing to infuriating.
“It’s really bad for the filmmakers and the actors, because it’s not the authentic representation of their work,” Mann says. “And as an experience, you’re not immersed if it’s not in synchronicity.”
Mann began investigating novel dubbing techniques. He explored head scans, but the rendering lacked realism. The dubbing merely moved from one uncanny valley to another.
Losing faith in established VFX, Mann started searching beyond the film industry. He soon stumbled upon a promising alternative: Deep Video Portraits.
Hollywood meets GenAI
Unveiled in 2018, Deep Video Portraits was a big breakthrough for the nascent generative AI sector. The technique enables photo-realistic reanimation of faces using just an input video. Each facial gesture and lip movement can then be synchronised with speech.
The life-like results stunned observers — including Mann. “It blew my mind,” he says.
Mann reached out to the research team. They agreed to collaborate on a new technical test: making De Niro’s character speak German.
The transformation, Mann says, was “like magic.”
“It was really understanding how a certain actor might say a certain line… You retain the performance, but you can alter the synchronicity.”
Mann believed the technique was ideal for Hollywood. To build the idea into a business, he sought advice from Nick Lynes, a tech industry veteran. Together, the duo co-founded Flawless in 2018.
The startup’s first product was TrueSync, a dubbing tool that studios are applying to Hollywood movies. Among them is Venom: The Last Dance, a Marvel blockbuster released last year.
Flawless also showcased a sizzle reel of AI-translated trailers at this year’s Cannes Film Festival. Still, not every client is ready to brag about the results.
Threatening acts
As the premiere of Fall approached, Lionsgate became anxious. GenAI was still a novel term back then, but unions were already concerned about the threats to performers. The studio feared the film’s visual dubbing would spark a backlash.
“They were going to pull the release if this wasn’t cleared up with Screen Actors Guild and there were mega nerves,” Mann recalls. “But luckily, we had planned for the consent workflows and [rights protections] early on.”
Flawless built the plan on several pillars. All the data would be legitimately sourced — rather than scraped without permission like so many GenAI firms do. Every output would be fully rights-cleared. The acting would remain true to the original performances. Any significant changes would require additional consent.
The startup also restricted the system’s operations. “We often call our models ‘narrow models’,” says Mann. “They’re large, but they’re focused on a specific aspect and curated for purpose. They’re very targeted and based on clean data that can be used for that purpose.”
Flawless presented the plan to the Screen Actors Guild (SAG). “They gave it the thumbs up,” says Mann. In August 2022, Fall was released theatrically in the US. The film — and the dubbing — were big successes.
Just a few months later, GenAI exploded into the mainstream. The trigger was the November launch of ChatGPT. A wave of image, text and audio generators followed closely in its wake. Suddenly, AI’s threats to actors, artists and copyrights had become public concerns.
Another take
In July 2023, the SAG-AFTRA actors’ union began the longest strike in its history. One of the guild’s prime concerns was the threat posed by AI.
After months of intense negotiations, the union reached a deal with Hollywood’s top companies. Under the agreement, any digital alterations would require explicit consent — unless they’re “substantially as scripted, performed, and/or recorded.”
Mann welcomed the terms. They wouldn’t curb lip synchronisation for foreign-language dubbing, but would enforce strict consent requirements for any meaningful changes to script or performances.
The new rules presented business opportunities for Flawless. By supporting union regulations, the startup hopes to rapidly gain traction in Hollywood.
A month after the SAG-AFTRA strike began, Flawless unveiled a new rights management platform. Named the Artistic Rights Treasury (A.R.T.), the system shares AI-generated edits with performers. If the actor approves the changes, they can consent within the app. If they don’t like the new versions, they can submit their own takes.
A.R.T has now been baked into DeepEditor. Mann believes the blend of AI editing and safeguards creates a unique product. “DeepEditor will be the first legitimate enterprise AI solution on the market,” he says. “Everything else is laced with controversy and rights issues.”
A better future for Hollywood?
Over time, Mann expects GenAI to unleash endless opportunities for filmmakers. He envisions shrinking costs, less drudgery, and lower barriers to entry. If all goes to plan, Hollywood will regain an appetite for originality.
“The key to this industry thriving is innovating — and embracing innovation responsibly,” Mann says.
Yet even he has lessons to learn about working responsibly. During production for the Fall sequel, Mann has run into a familiar problem.
“I accidentally have written far too many f*cks again,” he sighs. “We had to have a conversation: we’re allowed one f*k so let’s use it wisely.”
Get the TNW newsletter
Get the most important tech news in your inbox each week.