How Sydney creative studio Babekühl creates immersive live shows using Runway

Billy Ryan is a director at Babekühl, a Sydney-based creative studio and new art collective known for producing innovative music videos and live visuals for global artists such as Justin Bieber, Sam Smith, and TOKiMONSTA. In this interview, we speak to Billy about “Everything We Have Learned So Far,” a live show his team directed for Sydney’s Carriageworks venue as part of their work for MYSTICS, a collaborative project with Australian artists Jono Ma and Jonti.

Hi Billy! Thanks for taking the time to speak with us. Can you tell us a bit about how this live show came to be?

[“Everything We Have Learned So Far”] was an evolution of a performance that Jono Ma, Jonti, and Patrick Santamaria (from Babekühl) created in 2021, inspired by Blade Runner. During the conception of this show, we ventured into a new process of sonic and visual world building. This involved an intensive period of immersing ourselves in a conceptual and creative feedback loop of Babekühl creating moving images, and Jono and Jonti responding musically, and vice-versa.

Off the back of this show we were asked to develop an hour long performance for Vivid 2021. Due to Covid lockdowns it was unfortunately canceled; thankfully, it resurfaced in June as a newly conceived project we called “MYSTICS: Everything We Have Learned So Far.”

How did each artist collaborate with one another, and what did each artist bring to the project?

The collaboration was integral to the success. Process-wise the main thing that Mystics had that was different for us, was the collaborative means of creation. It was made more like how a band would jam to make music, except in this case we were making images and music. Very rarely do you work on something like this where you don't really have a clear picture of the end result but you head off in a direction that feels right.

How does your approach differ when working on live shows, as opposed to digital visuals?

For live shows, we try to take a more theatrical and holistic approach to it. Often we try to build a show around a three or four act structure, building as it goes towards some kind of climax or crescendo.

Additionally, similar to in cinema, the audience you have at a live performance is not likely to go anywhere, so you have the space and time to build up and ease into things. The surrounding is dark, and the sound and visuals can really hold an audience’s attention more easily.

Often with visuals online - content on socials, music videos etc., you have a lot of other competing distractions (e.g., the next video in the YouTube list). This means we often have a tendency to create something as eye-catching as possible, with a short window to grab the audience’s attention.

You used Runway’s Green Screen tool as part of your process. What led you to the software?

It was the combination of an Instagram comment we saw in a post, and a personal recommendation. I was also Googling things like “AI rotoscoping software” to find it, and the name stood out.

Tell us a bit about bringing Runway into your team. How quickly did you pick up the tools?

We have experience with Rotobrush and green screen plugins in Adobe CC and Resolve. Runway was super intuitive. As I recall, the point-and-click functionality was easy enough to understand without any training or tutorials. The speed at which it did the first rotoscope was unbelievably fast.

Like all good creatives, we are never that keen on a steep learning curve. More often than not we are learning new software and tools on a project in real time. Rarely do we have long time frames that afford the luxury of extensive training. So that entry curve of usability is so important.

How did Runway influence the creative?

The speed meant we were less concerned with perfection and more about the volume. For this project we were using the Green Screen elements as matte layers in After Effects to add animation within. It didn’t need to be perfect, but it was pretty close regardless. It allowed us to build a bank of silhouettes quickly and experiment with them en masse rather than spending a lot of time on one or two clips.

A few things we noticed was that even low-res footage ripped from YouTube rotoscoped pretty well in Runway. Stuff that previously we wouldn’t even try using the Rotobrush in After Effects with, we were able to use. In order to increase the size and add a bit of extra definition, we used Runway to rotoscope the elements we wanted from the background, then we’d bring those all into After Effects to recompose it.

You premiered “Everything We Have Learned So Far” on June 15, 2022. How did audiences react?

The response was tremendous. A large number of people who attended reached out afterwards to tell us how moved they were by it. I don’t really know what people were expecting when they came, as we were still very much tuning the concept right up to show time. This may have worked in our favor as it seemed to exceed any expectations and left a lot of people inspired and with a smile on their faces.

For us it was one of the most impactful and satisfying experiences of our creative journey so far. Hopefully it will resurface at some international festivals in the near future.

Find more work by Billy and Babekühl on their website and Instagram.