More and more — and increasingly so in a post-pandemic landscape — experiential design has become a key priority for cultural organizations hoping to deepen and broaden their offerings. The use of immersive technologies such as virtual and extended reality haven’t just transformed the ways in which visitors engage and interact with exhibitions, but how institutions embark on digital storytelling and preserve cultural heritage.

To better explore this expanding arena, the Los Angeles chapter of ACM SIGGRAPH, an international community of creative technologists, hosted “Museums, Immersive Experiences and Beyond.” The virtual panel, which featured innovators in the field, discussed the impact of immersion and what it might bode for the field of experience design. Here are three key ways immersion is offering cultural organizations new ways to engage their audiences.

Enabling digital storytelling

StoryFile’s conversational AI videos allow visitors to interact with recorded subjects in real-time. Image: StoryFile

For Sarah Kenderdine, Director of the Lab for Experimental Museology at EPFL in Switzerland, “socialization is the core of the museum experience,” and immersive technologies can be used to harness that socialization for good. Case in point: StoryFile’s series of recorded interviews with Holocaust and 9/11 survivors, which, using conversational artificial intelligence (AI), allow viewers to converse with the recorded subjects in real-time. In May, the Gilcrease Museum’s The Legacy of Survival exhibition featured StoryFile’s interactive testimonies, recorded with two survivors of the Tulsa Race Massacre.

It’s through these conversations that the technology company is hoping to deepen visitors’ engagement: “Though we had preserved stories in many forms,” said CEO Heather Smith, “we had not yet preserved that conversation.” More so, it demonstrates how technologies such as AI can enrich historically important storytelling through direct and immediate interactions.

Making data immersive

The Atlas of Maritime Buddhism touring exhibition features technologies like the panoramic navigator, which enable visitors to direct their experience of a site. Image: Fo Guang Shan Buddha Museum

Key to any immersive experience are the visuals on offer. For Kenderdine and Jeffrey Shaw, Dean of the School of Creative Media at the City University in Hong Kong, data, while not usually the most appealing of displays, can potentially be transformed by creative technologies into an interactive experience.

Recently, the duo collaborated on Atlas of Maritime Buddhism, an immersive exhibit currently touring China that documents the maritime routes through which Buddhism spread between and around India and China. To display footage, the show uses immersive technologies including a linear navigator, a panoramic navigator, and the iDome. With the navigators, viewers can control a camera to scroll or walk through a site, while the iDome allows them to experience moving film in a spherical immersive environment. “The malleability of digital data, its intrinsic quality,” noted Kenderdine, represents a well from which organizations can tap as they approach immersion.

Preserving cultural heritage

The 2012 exhibition Pure Land immersed visitors in Dunhuang’s Mogao Caves through projections and augmented reality. Image:

To illustrate the crucial role of digitization when it comes to cultural heritage, Kenderdine pointed to the Mogao Caves in Dunhuang, China. Housing cultural relics and art that date back more than 1,000 years, the UNESCO World Heritage site is at risk of deterioration wrought by climate change and foot traffic. It’s why photographers, armed with 3D scanners, have been carrying out a mass digitization of the caves.

In 2012, Kenderdine and team also unveiled Pure Land, an exhibition in which visitors engaged with the caves in augmented reality within a panoramic stereoscopic projection theater. For the Dunhuang Caves and other culturally significant sites, digitization may indeed become the most important tool for preservation, while enabling an international audience to experience the location without disrupting its environs. 


Jing Culture & Commerce