Spoiler alert: Yes, it’s already happening.
Scorsese. Kubrick. Lynch. Anyone with the last name Coppola. Iconic directors invariably have a distinct style and vision that make their work easily recognisable. Chances are that even if you didn’t see the words ‘Directed by Quentin Tarantino’ appear in the credits of a film, you’d still be able to identify the auteur through the kind of snappy dialogue, hammy violence and unique overall aesthetic seen on screen. Or would you?
In 2021, the world saw the first example of a film directed by an artificial intelligence (AI) designed to replicate the style of Italian director and screenwriter Federico Fellini. The implications of this are huge…
Even geniuses have a finite life span. Known for iconic films such as La Strada (1954), Nights of Cabiria (1957), La Dolce Vita (1960) and 8½ (1963), Fellini died in 1993. Having come of age under the trauma of Italian post-war fascism, he was fascinated by the subconscious, and faithfully kept dream journals for many years to help him understand his own drives and decisions. As a result, his films were known for their strange, dreamcore blend of fantasy, baroque and earthbound imagery. It is a style that many have tried and failed to imitate — until now. In 2021 — almost 30 years after his death — a brand new Fellini film premiered at the Venice and New York film festivals. While three living directors have credits for their work on the film and accompanying documentary, the overall artistic vision of Campari Red Diaries: Fellini Forward (2021) comes from “Fellini” himself, courtesy of an AI designed by innovation production studio Unit9 to precisely replicate the director’s style.
How Does It Work?
Perfect for a pattern-seeking intelligence with an appetite for material, the highly-recognisable style and prolific output of Fellini make the director the ideal candidate for AI imitation. Speaking to PC Magazine, Unit 9’s Marc D’Souza said:
“We used AI algorithms on a selection of Fellini’s past movies and scripts in order to extract a rich data set and establish patterns that could ascertain Fellini’s signature creative decisions. Natural language processing, computer vision, and image processing were used in order to extract the data. Shot type, scene cuts, camera movement, objects per frame, facial emotion and action/dialogue syntax and structure were all analysed, as well as scenes and locations.”
Academics in art history, cinema, mathematics and data science were called upon for academic supervision, but how did the team teach the AI to direct like Fellini? Creating custom tools and algorithms to help the intelligence on its way, as well as deploying some existing off-the-shelf technologies in a new way (e.g. TensorFlow, PyTorch), Unit9 ultimately created an AI that could:
Analyse millions of frames from Fellini’s films and scripts to extract what makes them ‘Felliniesque’
Use collected data to generate new suggestions
Break down every script Fellini ever wrote according to style, context, nomenclature, juxtapositions of concepts, recurring themes/motifs, and so on, to help it create an original, yet Felliniesque script
Identify and classify different cinematic shot types
Detect and track multiple people and objects on screen
Detect emotion within a live camera feed
Work on multiple directorial challenges at the same time
Generate a sequence of camera shots and cuts and assess them against Fellini’s visual language style before shooting
This Fellini shot-prediction model was developed by Unit9 from scratch, then integrated into Epic’s Unreal Engine as a plug-in. In effect, the AI was there to act as a second unit assistant director and cinematographer.
Using AI Responsibly
This film is part of a burgeoning trend around extending the work of artists from beyond the grave. We’re all used to record companies releasing new albums of archival music from the likes of Kurt Cobain, Johnny Cash and Amy Winehouse, and now, in recent years, we’ve seen uncanny holograms of Whitney Houston and Frank Zappa take to the stage to the delight of fans.
While raising similar questions around the ethics of releasing and potentially profiting from artistic works without the consent of the artist, living or dead, these latest developments pose new questions: is what we’re seeing even really their work? And how can we create new works that honour, rather than exploit, the artist?
To ensure the work remains truly faithful to the vision of the artist, even the best AIs cannot act alone. They must be expertly supervised, not just by humans, but by humans with real-world, tangible connections to the life and works of the artist. Surviving members of Fellini’s original crew consulted on the project, as did Francesca Fabbri Fellini: founder of Fellini Magazine, protector of his legacy, and the director’s niece. D'Souza explains:
“As the ones who knew the director’s spirit best, they were able to validate the AI’s vision so that the final output could be perceived as being truly Felliniesque."
This isn’t the first time AI has made its way onto the silver screen…
AI in the Industry
From promoting efficiency in pre-production activities like script-writing, casting, scheduling and successful location scouting, to visual effects, promoting and even greenlighting films, AI is fast becoming the new darling of Hollywood.
In 2016, the world saw the first ever AI-created movie trailer for Morgan, a horror film by 21st Century Fox. Analysing the sounds, visuals and composition of hundreds of existing horror trailers, the AI selected scenes from the final cut for editors to stitch together, reducing a long, complicated process into something that could be completed to great effect within a single day.
Then, in 2019 it was announced that James Dean, who died in 1955, will be playing the lead role in a forthcoming Vietnam war film, Finding Jack — all thanks to a combination of AI and computer-generated imagery (CGI). Of the casting decision, co-director Anton Ernst said:
“We searched high and low for the perfect character to portray the role of Rogan, which has some extremely complex character arcs, and after months of research, we decided on James Dean. We feel very honored that his family supports us and will take every precaution to ensure that his legacy as one of the most epic film stars to date is kept firmly intact. The family views this as his fourth movie, a movie he never got to make. We do not intend to let his fans down.”
Fox has teamed up with Google Cloud to create Merlin, an AI-based learning programme. Merlin’s primary goal is to analyse trailers and discover basic trends in viewers’ preferences for various genres of films.
Elsewhere, Warner Bros recently announced a deal with Cinelytic, an LA startup that uses data and analytics to predict how a film will perform financially before it is made — or even greenlit. This can help studios make important marketing and distribution decisions — although whether that influence will spill over into the creative process remains to be seen.
For many, this dance between technology and creativity is the crux of the issue.
What Does It Mean for Creativity?
Technology and creativity have long been in cahoots.
According to IBM, by “specifying teaching parameters for creativity, artists have gone as far as using AI to design sculptures and create paintings that mimic great works of art.” This means that we can, for example, teach an AI to paint in an impressionist style, simply by showing it enough pictures of existing works and using an algorithm to help it learn.
Indeed, an AI-generated piece recently won first prize at the Colorado State Fair — much to the disapproval of fellow competitors. The New York Times reported:
“Jason M. Allen of Pueblo West, Colorado, didn’t make his entry with a brush or a lump of clay. He created it with Midjourney, an artificial intelligence program that turns lines of text into hyper-realistic graphics.
“Mr. Allen’s work, 'Théâtre D’opéra Spatial,' took home the blue ribbon in the fair’s contest for emerging digital artists — making it one of the first AI-generated pieces to win such a prize, and setting off a fierce backlash from artists who accused him of, essentially, cheating.”
Allen, in turn, urged artists to overcome their objections to AI, even if only as a coping strategy:
“This isn’t going to stop…Art is dead, dude. It’s over. AI won. Humans lost.”
It’s a dramatic statement, for sure, but one that perhaps fails to appreciate the role AI could play in sparking genuine human creativity…
University of Sussex cognitive scientist Margaret Boden, who also serves as an advisor at Stephen Hawking’s Leverhulme Centre for the Future of Intelligence, explains:
“Many examples of creativity involve learning and exploring in a hierarchical style. Neural and multilayer network systems can help us construct different frameworks to better understand those hierarchies, but there’s much more to learn and discover.
“If you have a computer that comes up with random combinations of musical notes, a human being who has sufficient insight and time could well pick up an idea or two. A gifted artist, on the other hand, might hear the same random compilation and come away with a completely novel idea, one that sparks a totally new form of composition."
Boden estimates that 95% of what professional artists and scientists do is exploratory and perhaps the other 5% is truly transformational creativity. A lot of the processes behind creative thinking are still unknown and Boden believes AI has a big role to play here.
This is what it all comes down to — using AI and similar technologies to enhance, not replace, the brilliant minds of human creatives.
The Big Question
The aim of this post-human, experimental approach to direction is to find out whether a director’s oeuvre can be not just imitated, but extended, when they are no longer around to yell “cut”. This will come down to just how well companies like Unit9 can leverage technology.
“What I personally love about this project is that it really sparks debate over the future of cinema and the role that AI will play in it,” D’Souza told PCMag. “Plus, part of the beauty of using AI is that so much is possible — things we as humans may not have even thought of yet.”
In the end, did the AI-directed film resemble Fellini’s work? It must have; Francesca Fabbri Fellini has given it her blessing. This landmark film has proven that this kind of artistic vision doesn’t necessarily have to die with the artist. Their style can live on forever — so long as post-production is strong, it doesn’t matter that the director is post-human.
That said, I can’t help but wonder if this application of AI will catch on beyond a novel experiment. Could it be the start of a bizarre new trend that will see true-born creatives left on the cutting room floor? I hope not, but I guess we’ll have to stay tuned to find out.
Comments