How will AI change the viewing experience?

Gary Hustwit, the filmmaker behind the innovative Brian Eno doc that is different every time it is screened, says streaming services will soon have the capability to show movies and films that change with each watch, allowing them to personalise their offerings and potentially revolutionise the business.

Amid myriad existential challenges facing the documentary business, filmmakers and producers gathered in Toronto this week for the 31st annual Hot Docs film festival to ponder the future of the documentary. And one film in particular had tongues wagging.

Eno, a documentary feature about pioneering musician, producer and songwriter Brian Eno, was created using generative AI technology that means a new version is rendered every single time it is shown.

The general framework of the film is the same – it begins and ends in the same way – but what happens in-between changes every time, with the AI dipping into a mammoth library of archival material to render something new.

The documentary, which made its world premiere at Sundance in January and its Canadian premiere at Hot Docs this week, comes from American filmmaker Gary Hustwit (Rams, Urbanized).

The director and producer argues the industry is headed towards a place where films and TV projects use generative AI to render different, more personalised versions for its viewer. It might be too early for mass adoption, but it is coming.

“The rest of the industry isn’t ready for this. The streamers don’t have the capabilities yet, but they will need to have the ability to have more dynamically generated films and shows, because it gives the [project] more agency and more capability,” Hustwit said during a Hot Docs panel titled Driving Docs Forward: Tech and AI.

Speaking later with C21, Hustwit said he has spoken with the streaming platforms and knows it is something they are considering, especially in the context of offering content that is more specifically tailored towards the end user.

“It’s something they’re interested in, because they can see the changes coming with generative AI and they recognise the need for the films and shows they’re streaming to be able to change, adapt and be more personalised,” he tells C21.

“They’ve gotten so much data about the customers and their viewing needs. So far they’re only really using that for recommendation engines and changes in how the interface looks, but it can be extended to how the movies actually play or how they’re created.”

This kind of generative AI technology would not be limited to documentaries, notes Hustwit, who says it can be applied just as readily to features or scripted series.

“There are so many ways that capability could be used, but right now even having the architecture for it to be possible on a big streaming platform is something that still needs to be innovated. That’s what I’ve been focused on.”

One application, suggests Hustwit, would be for users to create their own version of a feature film by using AI to change specific scenes or actors.

“For example, you could design your own version of a film that reflects the things you want to see, or something that you can then share with a friend. Like ‘hey check it out, I made my own version of Oppenheimer,’” he says. “And maybe it’s not radically different but there could be certain actors or certain takes or certain parts of the film that are changeable. I’m excited about building the architecture for that to happen.”

Hustwit also revealed that he is part of the Sora ‘Red Teamers’ – a group of industry members that have been selected to experiment with the text-to-video AI programme that shook the content business to its core when it was first revealed in February.

Like many in the industry, Hustwit believes the arrival of Sora will drastically change the business of film and television.

“In the next 18 to 24 months, things like Sora are going to radically change what we do as filmmakers. Specifically, things like stock footage and animation – it’s going to completely change that world,” he says.

“I’m on the Red Team of Sora. I’m testing it, experimenting with it, trying to break it, and trying to use it how I would use it.”

The unveiling of Sora has been greeted with concern and intrigue in equal measure, with the technology creating a world of new opportunities for filmmakers at the same time as potentially eliminating the need for certain roles in the content-creation chain.

Last week, Chinese company ShengShu-AI revealed its own text-to-video software, Vidu, and many more are sure to follow in what will likely become a tech arms race to develop the most sophisticated tools.

Hustwit acknowledges that there are myriad concerns around these tools, particularly the topic of how they are trained, but says they will change the industry whether people resist them or not.

“We’ve seen with Chat GPT over the past year and a half how much it’s improved, how pervasive it’s gotten and I think the same thing will happen with video. It’s a tool, you can choose to use it; you can choose to reject it.”

From a creative perspective, the filmmaker says the use of generative AI in his project helped him to “surrender” an aspect of creative control to the technology.

“Usually, we’re such control freaks about every frame and crafting the narrative arc, but this requires a little bit more surrender. You’re really crafting a structure in which the film makes itself. So the job becomes: how does that complement or yield a logical, engaging documentary every time, regardless of which individual scenes happen to come up or not?”

Hustwit emphasises that the use of AI in personalising the content delivered on major streaming services is not a small thing. It could change the viewer experience in ways that could revolutionise how we consume content both in the cinema and at home.

The change will come eventually, but it might take a major filmmaker or project to spark mass interest in this new form of content delivery.

Asked whether certain streaming services could introduce such innovations in the near future, Hustwit says: “I think you could do it right now, but you can’t do it at scale. And the other side of this is that you need the content, [not just the] capability. Like, if Christopher Nolan announces that his next film is going to be different every time you watch it, people would lose their minds and then the industry would shift.

“Part of what I’ve been doing, showing Eno at film festivals and talking about it, is saying, ‘this is an idea for a way to make films and watch films that’s different from the past 130 years. What else can we do with this?’”