Creating immersive virtual worlds in which we are free to move and make choices, Virtuality Reality represents a new medium that will oblige us to rethink from zero the way we tell stories and consume them. The viewer, once a passive receiver of someone else’s stories, comfortably seated on a sofa or on a cinema seat, will become the active protagonist of VR stories: he will decide where to move in space, he will be challenged with decisions to take. Even more importantly, such a medium, considered to be the strongest “empathy machine” ever invented, will challenge us emotionally, through the experience of being temporarily in someones else’s body, may it be a different human being, a robot, an animal or even a mineral, and looking at the world with brand new eyes. A true copernican revolution in the way we consume stories, and, conversely, the beginning of a n exciting new era for story-telling, characterized by no more editing, no guided movement, and experiences that will be fully multidimensional and multisensorial, and highly, highly emotional. How to tell effectively a story, under such new conditions? And what are the risks, associated with these intimate and immersive VR experiences that, scientists warn us, have the power to reshape the limbs of our brains? We met with Adina Popescu, a visionary artist in the field of new media whose practice has often made use of virtual and augmented reality (“VR” and “AR”), the impact that this technology, might have on the future of content, narrative-building, and nothing less than our perception of reality. In 2015, she founded Snowblack, a studio for top-tier scripting, creative direction, consultancy, production and distribution strategies. Snowblack uitilises mobile, interactive, VR and AR technology, shaping the future of how we experience content. Beginning her career as a philosopher, writer and director in Berlin, she learned from directors like Alexander Kluge and drew her inspiration from Herzog. Since then, her work has expanded by integrating new technologies into her films. Her artistic work has been featured at Art Basel, the Venice Biennial, the Palais de Tokyo in Paris, and at galleries such as Marlborough in NYC and Eigen+Art and Autocenter in Berlin, just to name a few.

What’s so special about virtual reality?

Virtual reality is the “ultimate medium” for the simple reason that it overcomes the “limiting frame”, that has always embedded the content of other visual media we know from the past: painting, photography, cinema, and TV. Immersive technology is an exciting pioneer landscape. We are finding ourselves in a similar moment in time as we were in the first days of film – when everyone thought it would be crazy to create or even dream of long-form cinema. People were just staging theatre plays and filming them, basically trying to convert the old medium into the new. In a way this is what is happening in the VR space right now – especially with 360 video. It is only every other century that a new medium comes along. It allows for boundary pushing, establishing a new visual language, and setting a new paradigm for what is to come.

What is the difference between 360 video and VR?

360 video is technically not VR. You basically use classically mounted film cameras with lenses; you film in 360 but maintain camera movement, then a software is used to stitch your footage into a seamless panorama. I like 360 video, as it is still very close to the logic of making music videos in the 80s. It’s not great quality wise, but is highly innovative and allows us to experiment. Though it is still not VR. What is coming at us now is an entirely different beast. The best way to explain it is maybe thinking of how a disruptive economy works in general. For example: you have automobile companies that are making cars better and better, and then there is TESLA or Google X – companies that are creating a computer on wheels. The same applies to the camera industry: you have companies that are making better and better cameras, lenses etc, then you have photogrammetry and volumetric capture — no longer cameras but computers, calculating the information of a volume or capturing a space and recreating it as a photorealistic, dense 3D model. And the difference is that you can now move through that space when you are in VR. It is a virtual reality, similar to this reality, ideally with as much agency as you have in this one. And this changes radically the way we think about story telling and film. You don’t watch something happing – you are in it. And you are affecting it by your mere physical presence in there. You are free to do what you want to do. This is where it gets tricky for directing and storytelling. No more editing, no guided movement. So where does the story go? This is my task and this is what I am working on at the moment.

Creating content for this multi-sensory and multidimensional medium must represent a totally new challenge for narrative building.

This medium is like music – everything happens everywhere, in synchronicity, so you must think of a story and an experience along the lines of spatial composition. You need to think in emotions, and how to affect every one of the senses rather than just sight and hearing. Maybe you even feel more than you see in a VR. It is a multi-sensory, multi-dimensional space; if you want to control what is happening in there by creating a linear story, you will fail. It just does not work. You cannot take an old format like a TV show or a movie and just place it into this new medium. “The medium is the message”, wrote visionary Marshall McLuhan. We are only now starting to feel what he meant by that! Your content is formatted by the logic of the medium you are in. Within an immersive medium, these conditions are radically different from creating content to be watched on a screen. I believe that content in the future will be mobile, comprised of a mix of what is considered today to be gaming, narrative storytelling, and co-creating within a highly photorealistic, responsive and generative environment.

I like the idea of a medium in which feeling is, for once, more important than sight – the most rational and dominant sense in our era by far.

That’s why I believe this medium speaks to people with complex or contradictory skill sets: lyrical people who see music, dream fashion, taste art and hear poetry, and who are not scared by the complexity of the science fiction-like technology behind VR.

You recently declared that ‘the future of this medium is female’. What exactly did you mean by that? “

If you want to create and “think” content for VR, you have to renounce control over the story: you have to hand yourself over to this medium. Here you don’t impose your personal vision onto the viewer; you don’t control the gaze via editing. On the contrary: here the environment is first. You have to be able to create a virtual environment that is capable of luring the viewer into a complex web of possible narratives and story lines, making use simultaneously of all of the viewer’s senses. Maybe it is seduction and fuzzy logic rather than telling us what to see. I like the idea of all those male directors out there who now have to discover their female, sensual side in order to be successful in the future of entertainment!

Can you give us an example of this new form of non-linear story telling?

One could be the HBO show Westworld, whose narrative of humans interacting and getting pulled into the story-world of the AI is structured in the same way we are scripting VR narratives. You basically create a responsive environment, and then create a story-world in which every single possibility of interacting with the characters you meet is mapped out. And every path you take in this world leads you back to the main narrative at one point, while you can always branch off and go on different story paths. Its like many stories with nodes that allow you to go in either direction; all narrative routes have to make sense and allow you to experience your journey in a meaningful way.

Which projects are you working on right now?

Together Virtuosity Entertainment, which is a VR content studio founded by Brett Leonard (director of the Lawnmower Man) and Scott Ross’ (ex-Lucasfilm general manager and co-founder with James Cameron of Digital Domain), I am working on a slate of large entertainment pieces that I have scripted and conceived.

VR is a medium that allows the viewer — should we say the “experiencer”? — to experience other identities, to be someone else and see the world through this person’s eyes. It has often described as the most powerful “empathy machine” ever invented.

Imagine you see a humanoid Artificial Intelligence. Imagine you leap forward, you open your eyes, and then – you are the AI. Imagine now finding your field of vision altered, your hearing altered: you can feel, see, and perceive things that extend beyond your human bodily limitation, and thus have access to a wider reality. By that you also start to understand the humbling limits of being human; you learn about how our perception of the world is just not everything there is. Alternatively, you could ‘body migrate” and take on the identity of a woman of color for a week, finding out for yourself what that feels like and how your encounters change just by the reality of how others perceive you. Doesn’t that sound like the most powerful empathy machine ever invented?

Which commercial applications do you foresee for this “empathy machine”? Will it help companies convince consumers to buy their products? Let’s think of fashion, for example.

I am not even thinking that way. VR and AR will become two sides of the same coin within something that I like to call “synthetic reality”. If data will not be on your screen any longer but everywhere around you, it is time to think about how advertising may use this upcoming medium in order to embed their vision and their voice into that. But if information and images become three-dimensional and multi-sensory, I would say that sectors like fashion and luxury brands will be the first ones to benefit, as no other medium has spoken so strongly to people’s overall senses. The future of communication is not limited to facts and visuals, but can create a sensation so powerful that it can reel people in on a whole different level. I can touch, see, smell, become part of the thing itself. This opens doors to retail, fashion, luxury brands and awareness movements that are unforeseen and unprecedented. I can’t stop thinking up scenarios every single day while using these tools.

Are scientists already studying the power of this medium as an empathy generator?

Sure. One of the pioneers in this field is Jeremy Bailenson, director of the Virtual Human Interaction Lab at Stanford University and co-author of the beautiful book Infinite Reality,  which is a compelling introduction to VR. He is probably the world’s top expert of the effects of VR on human behavior. By using virtual reality with his “patients” to create the illusion of “body transfer” – that is, taking on the body of another person – Bailenson has been able to verify the extraordinary power of this technology to combat prejudice and discrimination. By “transforming” his patients into animals, such as cows and sharks, and even into coral, Bailenson’s VR influenced their eating habits and consumption, instilling in them a feeling for animal welfare and the environment. We are talking about long-lasting effects over time, despite the immersive experiments lasting only a few minutes.

So yes, every negative stimulation can be turned into a positive one. You can influence people to bring out the worst or the best in them. What we do with this medium is ultimately our responsibility; tech is always neutral – what we use it for is what ultimately counts. Just to give you another example: Virtuosity Entertainment works closely with Nisha N. Money, MD, who works as the senior medical advisor for the Department of Homeland Security. Together they are developing tests in VR to even regrow limbs: this is the power that the medium has! It is basically accessing the power of our brain.

VR and AR are scary and crazy at the same time. Application for post-traumatic stress disorder (PTSD), phobias and just generally opening the gateways of perception – all of it is possible and in fact is being done already.

During 2015, this immersive technology became a powerful ally of non-governmental organizations, which have started employing the “empathy machine” in order to potentiate their awareness- and  fundraising campaigns.

  

Let me stress that this is mainly done by using 360 video. The New York Times use of this technology is a great platform for these awareness and empathy pieces. One of the first examples of this trend is “Clouds over Sidra”, commissioned by the United Nations. This short 360 video allows the viewer to experience a walk in a Syrian refugee camp in Jordan through the eyes of a little girl who is living there. In May 2015, Amnesty International UK launched its first VR campaign, an impressive documentary that teleports the viewer among still-smoking ruins of a street of Aleppo, just after a bombing destroyed most of the buildings. In 360°: Act in Paris, the documentary filmed by actor Jared Leto on behalf of the American environmentalist association “The Sierra Club”, we find ourselves onboard a canoe surrounded by a beautiful, wild Alaskan tundra, powerless witnesses to fast-melting glaciers. It’s a fact that reading or watching a movie about social or environmental topics – from racist discrimination and violence against women to the pillaging of natural resources – influences the behavior of people much less than if you allow them to experience such issues themselves, by immersing them. In terms of fundraising, these VR campaigns have proven to be more efficient. This shouldn’t surprise. Watching the world through the eyes of another person can be a life-changing experience, capable of breaking down the walls of indifference and making us more generous, more inclined to help each other, regardless of all geographic, ideological, religious and cultural differences. I’ve heard it said that “empathy is the new currency of the future”.

I have been a consultant to CEO and Chair of Conservation International Peter Seligmann, and have helped to get “Valens Reef” on the way – a beautiful exploration of the state of the oceans that the viewer can dive into. Currently I am working on a large prototype for climate change awareness, using affective surfaces. We want to get under peoples’ skin, and not just into their minds.

I can’t help but think this powerful tool could also end up in the hands of the wrong people.

Yes, I wonder if us humans are even ready for such a powerful technology. VR is not 360 video, as most people believe: it’s a perfect, generative simulation of a second reality that is as dense as this one, a reality in which we can interact, co-create, encounter. On the positive side, VR and AR turn the passive consumer of the past into an active, co-creative being, who is moving through space instead of sitting on the couch. The transformative potential for creativity, thinking, even training your body and your brain are amazing.

But the low road is obviously there — and the responsibility that content creators and producers have here is vast. For instance, here are some questions that I need to take into account in my work: will we be able to kill people in VR? I have discussed this topic with the creators involved with the first-person multiplayer shooter game “Call of Duty”, and we all agree that we are going violence-free in this. Victoria Alonso, the head of physical production at Marvel Comics, has committed to the same statement when it comes to their content production in VR. If VR has the power to alter brain paths and create new ones, then our responsibility towards a new generation of kids that will grow up using and interacting within these worlds is grand. Maybe the same rules have to apply in VR as in this so-called ‘base reality’? You should not kill people here, why should you in there? This is not about censorship. You don’t consider the law against sexual harassment to be censorship, right? Why would you consider such a law in VR to be censorship? Does artistic freedom still apply in a medium that has the power to create a second life?

The other minefield that I see with the emergence of this new technology is the ways we will distribute information and how we will collect and deal with metadata. In order to place holographic projections of any form of data and embed then into this reality, we need to track every single person’s movement, track their eyes, and of course their movement and behavior. We need to read their behaviour patterns in order to push personalized content. Not only is this the end of privacy and intimacy, but it also opens doors to all kinds of abuse. Plus: Do I want to live in a solipsistic reality that was created just for me? How do I discover new things, how do I share, how do I co-alter, co-script, co-create?

It is very important to think about these things. Otherwise we are just creating the digital Matrix here. I do not want to be part of this; the danger comes from specific marketing and corporate forces. In that case, I do hope we will have the choice between the blue pill and the red pill.

You are currently developing a piece set between Johannesburg and Mozambique, working with world-class artists and musicians. You have some interesting characters in there – all them have hybrid identities in a way.

This idea of empathy raises also the question of identity and its representation. In a time of full transparency due to social media and of full availability of all possible narratives, our ethnic, national and sexual identities are more confused than ever. Who are the voices that are redefining and rewriting our identities today, since The Black Panthers, Blaxploitation, since Muhammad Ali and Nelson Mandela? The South African band Die Antwoord is a great example on how growing up in the face of racial conflict creates a complex white identity in return. Within my work as a director I want to give my generation a voice.

Who do you have in mind, in particular?

I have in mind the Black and white and more so, the mixed kids out there, sexually and racially. Those that deal with a confusing heritage, with a radically changing world, with the eternal question of belonging, of identities and their expression in music. There is no Black or white, no British-Pakistani nor French-Arab, gay, straight, transgendered box to tick. We are what we are. To some we might look like freaks. It is more likely though that we are the future. Our medium is sharable, mutable. Sometimes it’s viral, sometimes it is VR. Not only because it is new and immersive; also because it allows for you to get closer.

I guess you didn’t need a VR experience to feel emotionally linked to “Clouds Over Sidra”. I have read that you have been a refugee yourself growing up.

When I was 5 years old, my father and I got onto a plane and left Romania for good. My mother had the incredible courage to illegally cross several borders into Italy and took off from there with the help of the International Red Cross, trying to find a country to give us asylum. This was still under Ceausescu’s dictatorship.

When being allowed to visit my father or even speak with my mom on the telephone, we were usually watched and our phones were tapped. Somehow my mom got from Italy to Germany and wrangled a job in Munich, working at the Fraunhofer Institute. After seeing no hope to get asylum for my father and I, she wrote a private handwritten letter to the president of Bavaria – at that time Franz Joseph Strauss. Totally random. He somehow read it, and within 2 months time, my father and I got on that plane.

How do you remember those years?

We had to leave everything behind, renounce Romanian citizenship and got to move to Germany. The revolution was just about to happen in Romania and the country was in shambles. The first years were horrible, moving every year, trying to get permission for my dad to work, at first living in these terrible places outside Munich. But then my parents worked very hard and things became good. So good that it allowed me to get the best education that they envisioned for me, which I am so infinitely thankful for. My parents always dreamt of living in the US, but never made it there. Which is maybe why I have been living there for so long and am still back and forth between NY, LA und Berlin.

Do you think your interest in VR — the technique par excellence for questioning identity — may be somehow rooted in these dramatic, probably even traumatizing pages of your biography?

I often wonder how this being de-rooted, this state of constant not-belonging, constantly being considered a second-grade citizen, got me to work with all that next-level futurist tech, directing and creating worlds in VR and AR. I guess what always fascinated me about VR is that it is a non-place after all, in which you can create the rules of your own world. I can meet with my friend who is in Tokyo while I am in Nicaragua, and we can both start composing music or drawing in the same space, using nothing but our bodies, sound visualization tools like RED PILL VR or Google Tilt Brush, using all of our senses all at once. In there we can co-create, no matter where our physical bodies are. The moment I liked the most in Spike Jonze’s genius film HER was when the VR character said that she would now leave Joaquin Phoenix and go to a place in which she can communicate post-verbally with Alan Watts and compose music in a non-Euclidian space. 

 

Published on DUST#10 Metanoia, November 2016