From 2D To 3D: What Aspiring VR And 360° Journalists Should Know

There I was on the moon, listening to melancholic piano tunes and the radio chatter of astronauts. Craters as far as the eye could see, the blackness of the universe all around. While my head floated in space, my feet were firmly planted on the floor of a Berlin expo centre. Personally, I have always found VR intriguing. Watching this ZDF documentary from 2018, Journey to the Moon, was the first time I realized: You can combine this with journalism?!

Yes, this was a while ago. And yes, not all that glitters is gold. A few years later, in 2024, I handed in my MA thesis in Cultural Anthropology about the challenges – and potential – of XR journalism, with a special focus on 360° video and VR applications. The findings I share here will be a) selected (because my thesis covered a lot of different ground) and b) based on expert literature. If you need to refresh your memory on what XR journalism is, check out some of our older posts about the different terms, production examples or the genre's history.

If you're thinking: Uhh... but XR is on a downward slope? You're not entirely wrong. However, people still write and care about immersion, one of this kind of journalism's core principles, even if the angles vary – like gaming, for example. Here at DW Innovation, we're definitely still interested in the topic too, and continue to find interesting projects that incorporate XR or 3D models, such as our current GenKI4Media 3D use case.

Now, let's dive in.

The 360° Camera Conundrum

Ah yes, 360° cameras. They can cause dilemmas for journalists during filming. Suddenly, you're no longer standing behind the camera like you would be for a 2D video because this one looks in all directions at once. You've got to decide whether to stand inside of the frame or hide. If you stay, you can also have yourself digitally removed later – but with quite a bit of time, money and effort involved.

Communication scientist Tanja Aitamurto asked immersive journalists how they deal with their seemingly inherent presence in 360° reports. In fact, she believes that removing journalists from 360° videos could be considered a form of manipulation. Though journalists want to enhance the accuracy of their reports through immersive technologies, they hide from the 360° camera to achieve a more "objective" representation of events (Aitamurto, 2019, p. 10, 12). It kind of makes sense at first – why would you hang out in the middle of the frame, right?

According to Aitamurto, journalists consider their absence from scenes necessary to maintain the illusion of accuracy (i.e., the "real" representation of events) – in line with the conventions for achieving objectivity in 2D journalism. Because normally, if they're not presenters or interviewers, journalists are absent from their videos. Rather than pretending that 360° footage is an authentic recording of events over which they have no noticeable influence and in which they do not appear, Aitamurto argues that journalists could also treat their 360° work as a constructed event that they made and of which they are a part (2019, p.12). So, good news for immersive journalists. You don't have to hide behind a bush if you don't want to.

What's more, not every idea for a story lends itself easily to 360° coverage, even if you think it should. The promise of immersive technology is to be right in the middle of events instead of just watching. But what if you're at a music festival and the crowd surges unpredictably while you're still setting up your camera? Not so easy to pull off. Want to zoom in on someone? Traditional cameras – usually – offer better options for that. 360° video is more suited to wide-angle shots and zooms in it often lead to motion sickness, particularly if you're wearing a headset (Oswald, 2019, pp. 132–133). As XR-experts Manuela Feyder and Linda Rath-Wiggins state, it goes to show how journalists have a responsibility to be aware of potential negative physical and mental consequences of immersive technology (2018, p. 91).

Conceptualising a Story in 3D

Moving on to computer-generated VR: Ever wondered how to plan an immersive story? The answer is storyboards! Usually, they're square sketches visualizing individual scenes. But here, you draw circles and then add images, actions to be performed, or enter degree values and angles depending on where you want users to look. Including an entire environment is definitely a challenge. But storyboarding helps as it gives journalists a sense of the three-dimensionality of their story, according to Feyder and Rath-Wiggins (2018, p. 69).

The traditional understanding of 2D image compositions and linear narrative structures is broken down in immersive storytelling. 360° and VR media creators must meet new requirements that didn't exist in 2D journalism, combine journalistic skills and technical understanding (Feyder and Rath-Wiggins, 2018, pp. V, 67–68). It can feel like a lot at first, and there are hardly any universal production rules yet.

Besides, most journalists can't CGI-animate or 3D-model a story themselves. Storyboards are a language everyone understands, they let programmers know what you actually want from them. It's teamwork. Depending on the project, you need journalists or storytellers, camera operators, 3D artists or – programmers (Feyder and Rath-Wiggins, 2018, pp. 69, 124).

Incidentally, when it comes to inserting characters or avatars into a story, it's irrelevant whether they look "lifelike". De la Peña claims in her immersive story Project Syria, the environment and characters sometimes look "cartoony", "[...] but, when experienced in virtual reality, the piece creates a strong sensation of being present on scene" (2017, p. 210). This brings us to a key point about VR stories: Technology can't replace good storytelling. The latter still matters most.

Make Them Look: How to Approach Interactivity

Immersive media creators are keen to guide the gaze of their users in a 360° space. This is one way of trying to counteract potential disorientation. Every angle, every perspective and position could have an impact on the experience (Feyder and Rath-Wiggins, 2018, pp. 65–66). When creating a story, journalists must therefore ask themselves questions such as: "When users are looking around in a relaxed manner, what is in their direct field of vision? What can they see to their right and left, above and below? What can they see behind them when they look over their shoulder?" (Feyder and Rath-Wiggins, 2018, p. 74).

Triggers are the most common strategy for controlling where users look in a 360° space. They’re set impulses that motivate the viewer to change their line of sight. One of the most popular ones is audio. Direct appeals work (e.g., "Look over there!" or "Come with me!"). Other acoustic stimuli, such as noises and music, are also helpful, writes journalist Bernd Oswald (2019, pp. 135–136). When editing VR, he continues, it’s important to ensure that users have enough time to view individual shots and to let sequences linger – longer than you would in a 2D video. Nonetheless, there should be incentives to let the gaze wander. Another way of doing that is including avatars or objects that move, encouraging viewers to follow them with their eyes (Oswald, 2019, pp. 135–136).

If too many triggers are set at once at the beginning of a story, consumers may drop out of it, becoming frustrated. Feyder and Rath-Wiggins state that users want to build a relationship with the content being shown to them. The starting point should be obvious: "This means offering clear 'points of interest' so that the user's attention can be channelled. Or, with this knowledge, consciously creating an 'overload' of impressions" (Feyder and Rath-Wiggins, 2018, p. 68).

Journalists vs. Users: Who Decides How an Immersive Story is Experienced?

Media creators design immersive stories by anticipating user behaviour and placing triggers based on their assumptions about how audiences will act (cf. Endter, 2020, pp. 66–67). As a result, journalists' beliefs about users and about how an immersive work should be experienced are effectively "scripted" into the VR stories as action frameworks (cf. Akrich, 1992, p. 208).

Often, media creators believe that if there isn't something exciting to see in every direction in VR, it's not worth it. At the same time, there's this conviction that users shouldn't miss anything important. It's a difficult paradox to resolve, especially since users can pretty much do whatever they want as soon as they put on their VR goggles. Journalists lose their "gatekeeper" function of being the sole guide through a story (Feyder and Rath-Wiggins, 2018, p. VI). Attempts to guide the user's gaze help preserve narrative accuracy (intended facts, meaning, and perspective set by the creators) but limit user autonomy. As Aitamurto argues, such strategies reflect journalists' efforts to regain control they previously held in 2D storytelling (2019, pp. 9–10) – even though user agency is a major appeal of immersive media (Oswald, 2019, p. 132).

In the end, users can be "invisible voyeurs" where events unfold independently of their actions (like in this piece by DLF Kultur about Stasi interrogations) – or "embodied visitors" to a narrative experience with more potential for interaction (Dominguez, 2017, p. 8). Allowing users more freedom can make it more challenging to control how a particular message is experienced though. To address this, game designers have developed techniques: "Often the trick involves making the player believe they have more freedom of choice [...] than they actually possess" (Arjoranta, Koskimaa and Siitonen, 2021, p. 143). As Feyder and Rath-Wiggins put it, the role of journalists as decision-makers, who decide which story is told, with what content, in what form, for how long it extends etc., remains intact (2018, p. 91). But again – users don't have to do what is "expected" of them, so it's difficult to say who really has "the upper hand".

XR in Journalism: More Evolution than Revolution

XR hasn't revolutionized journalism. It has quietly expanded what journalism can do. Just know that enthusiasm for new technology alone isn't enough to use it. The message or theme of the story must be worthwhile and technically feasible (Ünal and Dönmez, 2018, p. 147). If you want to work as an immersive journalist, you won't only have to learn some new workflows – which can be fun – you also have to consider when an immersive experience is the most effective way to get people interested in news (Dominguez, 2017, p. 9).

Sources

Images

Key Visual by Igor Omilaev on Unsplash.

360° landscape with northern lights by Bryan Goff on Unsplash.

Literature

Some quotes have been translated from German into English by the author of this blogpost.

Aitamurto, Tanja: Normative paradoxes in 360° journalism: Contested accuracy and objectivity. In: Jones, Steve et al. (Eds.): New Media & Society. Volume 21, Issue 1. A.o. Los Angeles 2019, pp. 3–19.

Akrich, Madeleine: The De-Scription of Technical Objects. In: Bijker, E. Wiebe and Law, John (Eds.): Shaping Technology/ Building Society. Studies in Sociotechnical Change. Cambridge (MA)/ London 1992, pp. 205–224.

Arjoranta, Jonne; Koskimaa, Raine; and Siitonen, Marko: Immersive Gaming as Journalism. In: Uskali, Turo et al. (Eds.): Immersive Journalism as Storytelling. Ethics, Production, and Design. Oxon/ New York 2021, pp. 137–146.

De la Peña, Nonny: Towards Behavioural Realism. Experiments in Immersive Journalism. In: Aston, Judith; Gaudenzi, Sandra; and Rose, Mandy (Eds.): i-DOCS. The Evolving Practices of Interactive Documentary. New York/ Chichester 2017, pp. 206–221.

Dominguez, Eva: Going Beyond the Classic News Narrative Convention: The Background to and Challenges of Immersion in Journalism. In: Kaplan, Frederic et al. (Eds.): Frontiers in Digital Humanities, Sec. Human-Media Interaction. Volume 4, Issue 10. Lausanne 2017, pp. 1–11.

Endter, Cordula: Assistiert Altern. Die Entwicklung digitaler Technologien für und mit älteren Menschen. Hamburg 2020.

Feyder, Manuela and Rath-Wiggins, Linda: VR-Journalismus. Ein Handbuch für die journalistische Ausbildung und Praxis. Wiesbaden 2018.

Oswald, Bernd: Digitaler Journalismus. Ein Handbuch für Recherche, Produktion und Vermarktung. Zürich 2019.

Ünal, Recep and Dönmez, Servet Can: 360-Degree Video News Reporting. In: Yegen, Ceren and Güz, Nurettin (Eds.): Media with its news, approaches and fractions in new media age. Berlin 2018, pp. 131–152.