Supplement for Chapter 8: Immersion: Multisensory and Imaginative Design Strategies
Insert for page 150: Multi-sensory design
Of course, it is not always possible to increase the number of senses involved. For example, if your team works and meets online or you are designing an entirely online conference (a task I am currently engaged in myself), then you are limited to sight and sound primarily. And even if there are opportunities to engage more senses in an experience, it’s also important to use strategic sequencing and timing. Because we rarely think about leadership in terms of experience design, a good general guideline is to strive to increase the number of sensory modalities included in your experience designs in order to unlock the possibilities for more impactful experiences. Let me give you a simple example of the amazing power of multi-sensory experience design…
Planetary Bodies:
What I Learned About the Power of Multi-Sensory Design from Astronomy
I was once working with a group of aerospace engineers who were making components for the James Webb Space Telescope. They were just starting a science communication program to share details about their work and inspire people about astronomy, at first internally within their own company (so even the non-science employees would become more invested in the important work) and then later reaching out to the community. Although these engineers were absolute experts in the advanced math and engineering needed to make the most amazing telescope in history, communicating their work to the public was not their forte. And the last thing a program intended to inspire people about science needs is a poor communication design that could actually discourage people from knowing about science — or worse, set them running in the other direction. Immersive experience to the rescue!
Through a series of workshops, my team and I led them through an immersive method for communicating basic astronomy to people in order to set the stage for the importance of their work. It was an innovative program called Kinesthetic Astronomy. Rather than learning about the arrangement and motions of our solar system through presentations (a rather passive experience) in Kinesthetic Astronomy, each person actually plays the role of Planet Earth within a scale model of the solar system (very experiential). Through body position, rotation, tilt, and movement, experiencers are not not taught — but are personally empowered to discover for themselves (through guided inquiry methods) — the direction of Earth’s rotation, the astronomical meaning of a day, a month, a year (which fewer than half of the U.S. adult population understands, according to recent research), as well as the reason we have seasons, why Venus is only seen at dawn and dusk, why the best launch windows for missions to Mars only come around every two years, and gain an accurate perspective on the awesome scale and distances of our interstellar neighborhood.
It was a very simple experience design with huge learning outcomes. It engaged multiple senses in a strategic and interactive sequence, including seeing, hearing, touching, and the kinesthetic sense. The engineers took the experience design, modified it to suit their purposes, and used it with tremendous success in their program.
In the end, it was a very immersive and captivating way for these professionals to share the passion of their life’s work. Why is this? At first glance it may seem rather obvious that this approach is just more fun than, say, reading about astronomy. But there’s actually much more to it. See these of Chapter 8 for a discussion of neural activation.
Insert for page 158: Narrative Transport - a powerful example
Let’s look at a masterful example that bridges fiction and real-world narrative…
The first twenty-five minutes of Steven Spielberg’s World War II drama, Saving Private Ryan, is an emotionally wrenching, shockingly graphic, and brazenly realistic depiction of the Allies storming the beach at Normandy France on D-Day in World War Two. In fact, the sequence was so realistic that it was rumored to cause many war veterans to leave theaters, and VA posttraumatic stress counselors saw an uptick in appointments in the months following the film’s release. It is a sequence unprecedented in film history, and earned Spielberg an Academy Award for best director. For audiences everywhere it transported them into an experience of what war is really like.
“I basically did a lot of research about D-Day,” Spielberg said in a behind-the-scenes documentary about the film. “Talked to veterans and got painted for me a picture that I literally did not believe, and didn’t know if it would be responsible putting that on the screen because I wasn’t sure I believed it.” Ultimately, he extended what was a brief five-minute scene in the original script, to a twenty-five minute sequence, “…because I wanted the audience -- I wanted everybody -- to feel the same as those green recruits that were just off those Higgins boats, who had never seen combat before, as 95% of them hadn’t. It was complete chaos and I was trying to put chaos up onto the screen as close to what they told me as possible.”
If you have not seen this sequence, brace yourself before you do. In the world today where we see a recent resurgence of war and violence, it is a reminder of lessons our parents and grandparents learned and passed on to us. Spielberg’s expertise in narrative transport does not merely tell us about it, it gives an experience of it.
Narrative transport makes otherwise inaccessible experiences suddenly available to us. So what makes the recipe for narrative transport work and how can we use it for designing transformative experiences? Continue on in Chapter 8 to learn more.
Insert for Page 160: Immersion, Cell Phones, and Social Media
Emergence in Immersion
Combining multi-sensory immersion strategies with imaginative immersion and narrative transport strategies is the sweet spot for designing transformative experiences. When we do this well as designers, we may see emergence of new and undesigned components of an experience.
In my work, in most cases, deep immersion in an experience results in experiencers seeing themselves in entirely new and surprising ways. They often see themselves as new and more capable agents of action and change in the world and in their own lives, as many of the stories in this book highlight. It is usually accompanied with feelings that they did more than they thought they could, or endured more than they thought possible, or achieved something they never saw themselves even trying for. This is when things go well, of course.
However, sometimes immersion results in some emergent negative outcomes. A great example of this is cell phones. Trying to get my kids off their phones is like trying to unbake a potato. Here is a device with a very limited multi-sensory experience for the most part (seeing, hearing, and touching only) and shallow ones at that, compared to some of the “real-world” examples we discussed earlier. And yet the experience is absolutely captivating — for kids and adults alike. It can dominate our time and experience, which is not always good. Why is this?
First off, using a cell phone in almost any capacity is a spotlight experience, demanding our narrowly focused attention. When we use cell phones we shut out most of our other senses and are captivated, a perfect example of multi-sensory engagement and narrative transport. And yet it is rarely very deep or rich as most apps offer only brief episodic experiences.
Also, most cell-phone-based experiences are highly social, which brings with it emotional involvement and additional imaginative narrative elements. We see the words of text appear on our screens and imagine the person on the other end. We watch a video on TikTok and imagine the worldwide audience watching and commenting along with us. We create content ourselves and imagine the disembodied people “out there” in the cloud, wondering how they will receive it.
And of course most apps we use are designed for short bursts of interaction that are rewarded with satisfying graphics, animations, noises, chimes, and music that trigger dopamine release — pleasure. In fact, for many people, NOT checking their phone for notifications every few minutes triggers a stress response. This in turn results in a cycle of “set-up and payoff.” The almost irresistible urge to check the phone builds (stressful set up) until we do check it and are rewarded (pleasure payoff). It can be both an addictive and exhausting form of immersion.
I’ve had to deal with the challenge of online interactions in all of my experience designs, and have discovered a very subtle but obvious emergent strategy for turning them into tools rather than distractions or competitors for attentional deployment. As I tell my kids, when you switch off your phone, you switch on the real world of multi-sense. And it’s true. Even my die-hard cell-phone-using children agree that “no phone zones” are important and enjoyable.
In many ways, the contrast of online experiences with “real life” experiences causes new and emergent appreciation for ‘living with our whole bodies’ and inviting ourselves to be more immersed in the world available to us. That is why many experience designers forbid online activity of any sort in their designed experiences. In many cases, this is a great choice. However, I have found that incorporating phone use, and the social and narrative capabilities they bring, into my design is more powerful than eliminating them.
Designing to take advantage of the benefits of phone use requires a shift from phone-centered immersion to self-centered immersion. That is to say, designing cell phone use and other online interaction into your experiences, but in service to the personal experience of the design, rather than being directed by it. This is easier said than done, but it starts with the experience narrative you offer.
By placing the experiencer at the center of the narrative of your design from the beginning, and encouraging the use of their phone and online activities in support of that narrative authorship, we can position phones as another narrative tool. Using them strategically for video and audio capture, posting and sharing with friends, family, and audiences beyond the design, recording personal journals, notes, and observations, connecting your groups through virtual bonding before-during-and after the experience (Something we will explore more in the Chapter 9), and many other uses — can all serve to enrich the experience.
When I led the Bayer International Teen Science Camps for several years, on day-one we would collaboratively create the roles and responsibilities for the camp — the rules. These were high-performing, tech savvy, creative and brilliant kids who had incorporated online interaction into the fabric of their lives and identities. I would start off by asking them, “how are we going to use our phones and tablets in this experience — and how are we not?” After the shock dissipated that I did not declare the entire experience a no-phone-zone, they enthusiastically created agreed upon guideline for using them as tools, with limits, to enhance their experience and share it with family and friends back in their home countries — with enormous relief and gratitude.
If you are not a digital native and did not grow up with the internet, carefully consider the emergent possibilities of blending online experiences and tools into your designs. Think of it as a designer discomfort zone for yourself.