Limbic Media

Limbic Media

Tag: interactive technology

Adventures in Perception: Aurora

Limbic Media’s Project Technical Lead, Jason Sanche, recently wrote about Aurora in a paper for a university course on Neuroaesthetics. Jason is finishing his Computer Science degree at the University of Victoria and couldn’t have timed this article better with our recent posts about multisensory technologies and their effect on brain development and behavior. The following was adapted from a series of papers exploring perceptual experiences inspired by the artists’ discoveries and insights with specific artworks, and in this case, Aurora.

Aurora QA Station

Aurora: An Exploration in Perception

This article is an exploration of my perceptual and aesthetic experience of Aurora, a software platform developed by Limbic Media to map intricate sound qualities to light. Aurora listens and recognizes subtleties of sound and displays sound as patterns and shapes within two- and three-dimensional matrices of LED light. Aurora elegantly visualizes music with the subtlety of a musician’s ear.

Sounds have incredible texture, depth and emotional resonance, but these facets of sound often go unnoticed. Input from other senses, thoughts, and emotions, especially with the proliferation of screens, continually eclipses our simple awareness of sound. Subtleties get sublimated into the background of our experience. However, the act of listening attentively realigns the mind with time in a  constant, steady, somatic-acoustic present awareness. By showing sound as light, Aurora leverages the domination of visual stimuli and brings attention to sound.

Aurora’s hardware technology is, as Marshall McLuhan would say, an extension of our senses. In the same way, Aurora’s software is an extension of our minds and its neural and perceptual networks. Aurora performs similarly to synaesthesia—a perceptual phenomenon in which stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway. We naturally create images when hearing sounds, which might be an inherited survival trait to anticipate danger from threatening sounds. Incredibly, it also applies to appealing sounds like beautiful music. With our eyes closed and our attention narrowed in on music and the field of mental images, a synaesthetic effect involuntarily transforms sound into mental imagery. Aurora extends this phenomenon into shared space.

The following is a subjective exploration of this idea as a written stream of consciousness about the experience of Aurora reacting to a piece of music:

As the music streams from its digital storage on a cloud of distributed data into the physical network of this room, through my computer, through the mixer and into Aurora which, in real time, processes the signal through digital filters, determined by code, transformed into patterns and colors imitating the harmonies, rhythms and beats, transmuted into bursts of photons varied by a full visible spectrum of color and coordinated patterns with the complexity of the wave patterns in the ocean and captured by my eyes, translated in the optical nerve back into electrical impulses, and again, in real time, perceived as what I believe is sentient to the experience and understood as meaningful. As I write and watch the dancing lights, making the music more beautiful, I perceive and write and know this harmony of embodied sensory experience augmented by technology-as-art.

The study of neuroaesthetics looks at how the mind perceives and attaches meaning to art, beauty, and ugliness, how we fixate on and identify value, and how art produces emotional reactions. A system like Aurora provides a rich and fascinating angle to explore interactivity in neuroaesthetics, and specifically how the perceptual feedback of sound visualization plays into the brain’s implicit synaesthesia.

When sound-to-image happens externally, how does that affect our internal imagination of sounds and music? How important is sound-to-image synaesthesia to our ability to thrive culturally and socially? Can technology like Aurora produce a shared synaesthesia similar to shared public experiences during a film, concert or theatre performance? In participatory public theatre like Sleep No More, the play creates an immersive experience by breaking down divisions between actors and the audience. Can Aurora similarly produce immersive shared participatory musical synaesthetic experiences? The potential is there.

 

Innovation Tree, Victoria, BC

Art’s Role in Imagined Embodiment

Imagined embodiment has been a common theme throughout my explorations in perception. The mind constantly reinterprets its sense of self and embodiment in the world through imagination and dreaming, and the habitual sense of self is usually reinforced if we are unconscious of this process. However, with the right attention and tenacity, we can have full control over our self identity and full freedom from its limiting influence on our inhibitions. Anyone can imagine themselves as anything or anyone, and with enough practice, anyone can act beyond their usual identity. Most people enjoy an occasional respite from the trappings of their identity through events like halloween, masquerade parties, games, and to some extent, books and films that transport us into characters we can safely identify with.

One important role of art is to challenge and disrupt habitual identity through the perceptual experience of imagined embodiment, and made possible by mirror neurons. Conscious engagement in this process through art can introduce viewers to new horizons in self-knowledge.

PGNB Prismo 2017

Interactive Digital Art and Synaesthesia as a Method of Embodiment

The most relevant method of imagined embodiment to Aurora involves the exploration of synaesthesia. Synaesthesia is a unique doorway from the visual to the aural. If we pay attention to sound and its effects on the imagination, it has the potential to create a transformative experience and disrupt habitual sensory perception. Experiencing synaesthesia consciously by meditating on music or sound and absorbing mental imagery restores attentive listening and its meditative benefits. Interactive digital art like Aurora uses technology to leverage synaesthesia and bring audiences back to the present through attentive listening.

Aurora’s lights react to sound the way our mind would visually imagine the source of any sound. The nature of sound and the act of listening have a unique quality that visual perception does not—sound disappears nearly as soon as it is heard; it is more ephemeral and decays quickly through friction unlike most visual objects, which tend to persist until they are destroyed, or decay over longer periods. Because sound does not persist very long, attention to sound created a synchronization to the ineffable flow of time: the steady, consistent arising and dissolving of the soundscape. By perceiving sound as light we tune into the act of listening which is an important way of staying balanced and present in a sensorially fractured world. This allows the mind to be present in time, which is its natural state.

 

For updates on new content, sign up for our newsletter.

From Education to Branding: The Rise of Multisensory Experiences in Technology

Once upon a time, Redcliffe Elementary School in South Carolina was experiencing issues not unlike many North American public schools. Despite their focus on achievement in standardized testing, Redcliffe had the lowest test scores in their district. Instead of beefing up their academic approach, they decided to invest more in multisensory education that amalgamated traditional studying with visual arts, music, and dance. Within six years, their test scores were in the district’s top 5%.

The idea of integrating all five senses into learning is not a new idea, but the market for multisensory technology is quickly expanding as we more deeply acknowledge how it enhances human experience. Multisensory technology has played a big role in our discussions on heads-up vs. heads-down technology and how each influences us socially. While there are benefits to both, heads-down technology seems to dominate our consumption. There’s an underlying fear of the effects an imbalance of heads-down stimulation has on our behavior and cognitive ability, especially among children.

If we want product design, business models, educational tools, and social platforms to create a balance, multisensory technology offers solutions. Why are multisensory experiences so attractive to businesses and consumers looking to fill that niche?

 

What is research saying about multisensory experiences?

Most of us have had this experience at some point: you catch a whiff of something that sends you reeling in visual, emotional, and perhaps even auditory memories surrounding that smell. Our most established and vivid memories are multisensory. We form memories based on situational context, and that includes cues from all five senses and any relevant emotional associations. The growing popularity of multisensory experiences in everything from autism therapies to retail spaces is based not only on its inherent appeal and entertainment value, but a body of research focused on multisensory experiences and their ability to form lasting memories. The most logical place to measure the effect of multisensory stimuli on memory and learning is in childhood classrooms.

There’s plenty of literature out there to support the use of multisensory activities in classroom learning environments, especially from a young age:

  • A study that trained writing students with audio-visual methods looked at the effects of multisensory education on children with dyslexia. The study found that the approach improved students’ performance whether or not they were dyslexic.
  • Teaching letters to children and measuring their phonemic awareness was much more effective when researchers added a visual-tactile component to their delivery.
  • Incorporating full-spectrum lighting and changing colour schemes into learning environments has the potential to reduce stress and enhance student focus.
  • Playing music in conjunction with lessons tends to improve spatial-temporal reasoning, which is useful for understanding proportions and geometry.

People retain information better when they’re educated with their individual learning style. Multisensory education has so much success because it single-handedly addresses a variety of classroom learning styles, increasing the chance that lessons will be retained for each student. By addressing multiple learning styles, it also increases the variety of neural pathways stimulated in the brain, which is important for early brain development and learning. Classrooms that incorporate more multisensory teaching methods generally see higher rates of comprehension than ones that use uncoupled senses in their curriculum.

The issue of comprehension came up in our last post as well. Technology, especially tools like the internet, is thought of primarily as a heads-down influence. Even though this is often regarded as a negative thing, research has pointed out its ability to train our brains to sift through information quickly and jump from one piece to the next more efficiently—but this comes at the expense of in-depth comprehension. Multisensory approaches are encouraging us to rethink the role of technology as an aid to vastly improve comprehension rather than diminish it.

 

How are these discoveries affecting the market?

It’s no surprise that multisensory experiences are increasingly utilized by companies who understand how to infiltrate consumers’ memories and emotions. Consumers also invest more of their time and money than ever into experiences rather than consumer goods. As a result, brands are integrating multisensory technologies into their products and marketing campaigns, especially in brick-and-mortar settings. The future of retail, entertainment, education, and even some forms of therapy lies in multisensory technology.

A 2015 survey by the Event Marketing Institute found that 98% of consumers surveyed were more likely to purchase a product or service if it was marketed through an experiential, multisensory campaign. 81% were motivated to visit an experiential marketing campaign because of its potential to give something back to the consumer. The Institute also found that on average, companies will increase their experiential and multisensory marketing campaigns by 6.1% in 2015; a number that has likely increased in the last few years. There is still huge potential for multisensory experiences to grow in brand marketing. 48% of Asian marketers use a multisensory approach, followed by 28% in North America and only 13% in Western Europe.

People are more invested in multisensory technology and experiences over material goods because of its ability to enhance deeper connections, memories, and learning in our distraction-laden and isolating world. The more brands, educational institutions, and other influential sectors pick up on this trend, the more multisensory technology will become an essential part of everyday human experience.

 

What else does multisensory technology have to offer?

This article has focused mainly on the benefits of multisensory experiences in terms of research and the implications that research has on educational models and marketing. There’s a plethora of reasons for the rise of multisensory technology that don’t necessarily require the justification of research, though. Multisensory technology provides avenues for public, heads-up social engagement. It creates opportunities to form communities and transform underutilized public space into social hubs. It adds unique components to products and services that people don’t normally experience at home, and it creates a niche for various sectors, like shopping malls or civic bodies, to increase the return of their investments both fiscally and culturally.

Both research and mainstream media’s rhetoric on technology’s social influence seems to agree that “heads-down” technology favours certain reflex-based skills (hand-eye coordination, fast-paced problem solving, visual-spatial processing, to name a few) at the expense of deeper comprehension, connection and focus. Multisensory technology is unique from that experience alone. It satisfies the same skills that heads-down technology offers by offering instant gratification and an abundance of stimuli to process—but it also captivates us, holds our attention, and enhances our ability to learn and make social connections. In other words, multisensory design brings out the best parts of how we consume technology.

 

Looking for a multisensory solution to a project? Contact us today to learn more about Aurora.

Heads-up vs. Heads-down Technology: Impacts On Brain Development

Nicholas Carr

Photo: Antenna

When it comes to brain development, technology gets a bad rap. Nicholas Carr, author of The Shallows: What the Internet is Doing to Our Brains, compares our new ways of interacting to a backwards evolution: “We are evolving from cultivators of personal knowledge into hunters and gatherers in the electronic data forest. In the process, we seem fated to sacrifice much of what makes our minds so interesting.”

Last week’s blog tailed off with a different comparison. We suggested the connection between heads-up technology, which forms genuine social connections, and how those technologies reflect early childhood experiences. This week delves more into the question of how technology impacts brain development, especially when parsed by what we like to call heads-up and heads-down varieties. This is pertinent among children who are now spending their formative years engaged with technology. Technology observably impacts our social habits, but more easily overlooked is how it physically rewires our brain.

 

Child Brain Development

Photo: The News International

How is technology “rewiring” our brains?

The speed at which technology is changing and entering our world means that research on how it psychologically influences us can’t possibly keep up. But one thing is likely—technology is rewiring our (and especially our children’s) brains in ways that we haven’t yet encountered and don’t fully understand.

A 2007 UCLA study measured brain activity of experienced vs. non-experienced web-users in their prefrontal cortexes, areas associated with problem-solving and decision-making. The study found localized brain activity in experienced users much higher than their counterparts, even though brain activity was comparable across all participants when exposed to non-internet-based reading tasks. After instructing the non-users to engage in internet use for one hour per day for six days, the study was repeated and found comparable brain activity across all users.

Even though this is only a single study in a large body of research on technology and brain activity, it just goes to show the incredible plasticity of our brains and how quickly technology exposure has a physical and yet subconscious influence on our social and sensory experiences. It’s also worth noting whether studies of this type differentiate what we’d consider heads-up or heads-down technology; this particular study on internet use would definitely fall under the latter type.

That isn’t to say that internet exposure, or technology in general, is necessarily rewiring our brains in bad ways, just different ways. Using technologies like the internet has a tendency to frequently redirect our attention, forcing our brain to spend energy reorienting itself at the expense of comprehension. These sacrifices are known to researchers as switching costs. Considering the overload of advertising, hyperlinks, and other visual re-directors, it’s no surprise that humans are cognitively paying a higher price for switching costs than ever before.

 

The pros and cons of heads-down technology on brain development

If the influence of technology burdens us with switching costs, what are the benefits of technology, especially of the heads-down variety, in rewiring our brains? Nicholas Carr, a fan of Neolithic metaphors, describes the cognitive skills brought on by internet consumption as “primitive mental functions:”

  • Hand-eye coordination
  • Reflex response
  • Visual-cue processing
  • Fast-paced problem solving
  • Credibility assessment
  • Pattern detection

While these influences have their benefits, they are seemingly more base than socially interactive in their enhancement of human brain function. Evidently, a balance of the two is ideal for healthy brain development, especially in children. Patricia Greenfield, a developmental psychologist at UCLA reviewed over 40 studies in 2009 to assess the effects of technology and media on intelligence and learning. According to Greenfield, the “widespread and sophisticated development of visual-spatial skills” has come at the expense of “deep processing,” “mindful knowledge acquisition, inductive analysis, critical thinking, imagination, and reflection”—skills that, one could argue, are more relevant to forming our social tendencies early on in life.

 

Balancing technology and brain development

We’ve already discussed how heads-up technology is designed to encourage face-to-face interaction in social settings. It engages users with their surrounding environment rather than isolating their attention spans. In terms of technology rewiring our brains, there are parallels between heads-down technology and the visual-spatial skills it enhances, and heads-up technology and the “deep processing” skills Greenfield speaks of.

In an ideal world, humans would have the self-awareness and control to use both technology types in healthy amounts and encourage the same use among their children. “Heads-up technology” is only a novel idea because it is usually the exception rather than the norm, however. Heads-down tech just seems to be more widely accessible and leaves us more susceptible to unhealthy, addictive tendencies. If we hope to avoid the detrimental effects this has on our (and our kids’) brain development, there needs to be more research on what separates heads-up from heads-down technology and how the use of each is “good” for brain development.

 

Photo: Super Glam Moms

Multisensory technology and brain development

One area of research that studies the influence of what we’d consider heads-up technology on brain development focuses on children with Autism Spectrum Disorder and multisensory experiences. One such study, conducted at the Albert Einstein College of Medicine at Yeshiva University, found that children diagnosed with ASD do not process multisensory information as effectively or quickly as other children in that age group. This kind of research points to why there is a burgeoning industry of multisensory experiences and therapies geared toward families living with autism. The Children’s Museum of Atlanta, for example, hosts a regular multisensory program. One of the Yeshiva study authors went as far as to compare multisensory integration therapies to a “military-industrial complex.”

While there is definitely more room for research on the direct impacts of multisensory and heads-up technology on our brains, perhaps the approach’s popularity has more to do with the impacts of heads-up technology regardless of a diagnosis like ASD. Multisensory tech encourages our brains to fire on all cylinders rather than tune out certain aspects of brain development, and could be good for encouraging critical thinking and focusing attention in young brains that respond well to non-conventional learning styles. Continued research in this area will help us decide how to balance our exposure to various kinds of technology and approach product design, especially for children.

 

This week’s article evolved into what feels more like a research paper than your average blog, closing with more questions than solutions and answers. We know relatively little about the long-term effects that technology consumption has on the human brain, and we know even less about those effects when you try separating technology into categories like heads-up and heads-down. At the end of the day, a healthy dose of both types is likely ideal for well-rounded brain development, and we’re seeing a growing emphasis on engaging, multisensory technologies to influence that development, especially among children and those with disorders like autism.

One way of delving further into this topic is by looking more specifically into the science behind multisensory experiences. Public spaces are investing more into services and displays that captivate and engage audiences on auditory, visual, tactile, and even olfactory levels. Tune in next week to find out more about what’s behind the growing appeal for multisensory technologies.

 

Don’t forget to sign up for our newsletter for more great content!

 

Which Technologies Enhance Human Connection?

“What gunpowder did for war the printing press has done for the mind.”

—Wendell Phillips

 

No matter the era, technology has both a positive and negative impact on our lives. It all depends, of course, on how it’s used.

Most of America, apparently, would agree with that statement. When asked what the Digital Revolution’s impact has been on the overall quality of life in a 2015 Heartland Monitor poll, 62% of our participating neighbours believed that it has a mixed positive and negative impact. Less than half felt it was socially isolating and detrimental to forming communities.

It’s hard to measure the true impacts of technology, particularly social platforms, but there is undoubtedly huge potential for technology to benefit our connectivity as social creatures. What types of technology enrich our social connections? Over the next few weeks, Limbic Media will discuss how technology influences us socially.

 

Cellphone UseHeads-Up vs. Heads-Down Technology

Opinions in that Heartland poll were, unsurprisingly, more divided when parsed by demographics such as age, education, employment status and whether or not the participant had children. Even with a subjective question like “does technology help or hinder us,”  people still have very straightforward opinions about how tech negatively affects relationships, whether app addiction affects our social life or the danger of distraction has an impending threat to our survival.

Most conversations about human connection and technology, however, seem to lump “technology” into a single category. Evidently, not all technology affects how we interact with each other in the same way. It really comes down to the specific product, and again, how we decide to use it.

Since we don’t have time to discuss the effect of individual products on human connection, we can simplify things by describing most social technology as “heads-up” or “heads-down” (or a combination of both). Heads-up tech focuses our attention on our surroundings and encourages shared experiences, while heads-down tech tunes out our surroundings. Both types are beneficial in the right doses and settings, but for the purpose of expanding real-life human connection, heads-up is the better approach.

 

Brian Solis

Photo: Leading Authorities

Shared Experience Models are Gaining Traction

More companies are engaging with heads-up technology and shared experience models to satisfy customers’ needs for genuine connections. Thinkers like Brian Solis, author of What’s the Future of Business? are emphasizing the presence of shared experiences in company products, services, and internal relations: “No matter how much or fast technology (social, mobile, real-time) is thrust upon your markets, the one thing that remains constant is that people will use it to connect with one another, learn, and discover, create and curate, and most importantly, share and feel experiences.”

Technologies that serve our need for connection in positive and real ways not only benefit the public, then, but also the longevity of companies that understand the importance of social connections in their business models. We can all benefit from products that force heads-up connections by virtue of their design—what are some tangible examples of companies and products that embody this approach to social technology?

 

Aurora Jam Tent-Discover Tectoria 2018

Photo: CrackerJackFlash

Aurora and Social Wearables

Limbic Media’s very own Aurora (released last fall) and Social Wearables (yet to be officially released), are both designed as heads-up social platforms. Aurora, the world’s most advanced sound-to-light mapping platform, uses interactive sound, light, and technology in public spaces to encourage social engagement through art. Anyone with the free Aurora app can connect with the product’s lighting design AI and control how light shows interact with its audience in real-time. In future releases, Aurora will be able  to respond to motion and social media hashtags to influence lighting effects (#blue to change light colour, for example). Aurora is a heads-up social technology that can apply to a variety of social settings.

Currently in development, Social Wearables acts as a digital icebreaker. It’s designed to enhance networking opportunities and encourage face-to-face connections in social gatherings like conferences. Social Wearables is a light pendant coloured with RGB LED Lights; when wearers touch their pendant with ones of different colours, it vibrates and collects new shades until they Capture the Rainbow, the first in an upcoming catalog of games. The Social Wearables technology gets people’s attention up, brings people to you and provides a social context for approaching someone you’ve never talked to before.

 

Photo: Colossal

Interactive Seesaws

Cities like Montreal and Chicago have seen the appearance of interactive seesaws in their city centres. Impulse acts like an urban instrument; the weight and motion of see-sawers create a totally unique composition of sound and light for each duo. The multisensory creation is emitted from each seesaw with speakers and LEDs, and projected in real-time onto surrounding buildings. The heads-up installation is a collaboration between Toronto-based Lateral Office, Montreal’s CS Design and engineering EGB Group. The seesaws are a good example of taking an age-old social platform—playgrounds—and using technology to reinvent it into a public social experience for all ages.

 

Carrot Reward Program-Phone

Photo: Healthy Families BC

Heads-up Apps

Aurora and Impulse are good examples of how heads-up social technology is changing our approach to public space, but what about heads-up technology you can fit in your pocket? People’s desire for more genuine connections is making an impact on app design. Apps that encourage social interaction, physical activity and just plain old getting off your phone are becoming more popular.

The rise in anti-app apps reveals people’s awareness of their overuse of heads-down technology. Apps like Offtime, Moment and Flipd provide users with analytics on their technology consumption, especially with social platforms like Facebook, and encourage you to focus on other social activities and tasks.

There’s also a vast selection of heads-up apps that act as fitness trackers, designed to encourage social connections through physical activity. Hotseat is a good example of a physical activity app that recognizes the importance of short, frequent breaks at the workplace. Its goal is to to get employees away from their screens for short bursts throughout the day and enhance colleague relationships through exercise and competition. Similarly, Carrot connects friends to create collaborative fitness goals. Successful challenges are met with rewards points for experiential programs like SCENE and Aeroplan Miles.

 

Interacting with Pokemon Go

Photo: The Arabian Marketer

Augmented Reality Apps

Pokémon Go’s popularity spurred conversations on how augmented reality games and apps reflect our social tendencies. Even though Pokémon Go encourages players to get outside and interact face-to-face with players, a UBC study found that the least successful players tend to self-identify as introverted and socially awkward. The research points to a niche in gaming or other apps—adapting augmented reality to maximize social opportunities, especially for those that struggle to make genuine connections. The most widely used AR apps certainly help people interact with their environment, but there is huge potential for augmented reality to target users that are seeking social interaction specifically.

 

Technology has made our world more connected than ever before. Ironically, the ease of that connection has left many feeling no more genuinely connected to others. Perhaps the most satisfying aspect of life is obtaining meaningful human connections. Engaging more with heads-up technology over tech that socially isolates us helps bridge the gap between the quantity and quality of our interactions.

People resonate with technology that fills the need for real, tangible connections rather than shallow ones. There seems to be an overarching theme with products and businesses that emphasize these shared experience models. From Seesaws to Pokémon Go, we are attracted to heads-up technology that mimics childlike ways of interacting with the world. Children aren’t as capable of tuning out their surroundings; they are field experts at creating connections with the world with zero barriers. Companies that are looking to incorporate more shared experiences into their products and services might need to consider social technologies that can be appreciated by kids and adults alike.

 

Subscribe to our newsletter for more great content!

8 Interactive Technologies of Future Shopping Malls

When Rebecca Minkoff’s flagship store opened in SoHo in 2015, the retailer was known primarily for selling bags. Since introducing interactive technology for their customers on the floor, clothing sales tripled their expectations within a year.

Interactive technology in retail has been the driving theme behind our previous two blogs on shopping malls. We detailed how the offline retail industry has changed over the last decade and how it can adapt, in or out of lucrative holiday seasons. In addition to the impact of e-commerce and economic downturns, people’s spending habits have been influenced by an increasing desire for multisensory experiences, often in offline spaces.

Investing in interactive public art has a huge ROI both fiscally and culturally for their surrounding communities and businesses. Establishing a returning audience through these experiences is the answer to shopping malls’ survival. What exactly do these experiences look like in today’s malls? In the third and final article in our shopping mall series, we’re going to look at six different technologies you can find in the world’s most cutting-edge and successful shopping malls.

 

Photo: Marina Nazario/ Business Insider

1. Interactive mirrors

Oh, the drudgery of standing in line for a changing room only to be harassed by a sales associate as you struggle with a top—brands are now using interactivity to make changing room experiences fun and unique. Ralph Lauren’s flagship store in Manhattan implemented touch-screen mirrors that display your items and let you adjust the lighting. You can also request different sizes via touch-screen from your sales associate, who lets you know in real-time when they’re en route.

Touch-screen mirrors are a good example of how retail spaces are mimicking the interactive aspects of e-commerce. Rebecca Minkoff’s store houses interactive mirrors with eBay’s inventory management software. The mirrors act like virtual personal style assistants, making suggestions and telling customers exactly what’s in stock. The interactivity also helps the company track spending habits while increasing sales.

 

Photo: Business Wire

2. Virtual try-on

Less widespread than interactive mirrors is the virtual try-on mirror released by Samsung in 2015. The idea is to set up mirrors in non-retail spaces that provide customers with a hassle-free, interactive way to engage with products without stepping foot inside a store.

A  similar product was more recently patented by Amazon, and acts like a full-body Snapchat filter that integrates virtual try-on with backdrops in various locations. The company claims to use the world’s most advanced technology in light and projection to bring online shopping models to an offline, participatory audience.

 

Photo: brandchannel

3. Interactive window displays

Bloomingdale’s in Manhattan installed an interactive window display for Father’s Day in 2015. The goal of the project was to stop foot traffic by offering a captivating participatory experience. It also provided an opportunity to make sales out of convenience; if customers liked the products they experienced, they could make purchases without taking extra time to go inside.

Similarly, French shoe retailer Repetto drew in foot traffic by using motion-tracking technology in an interactive window display. Audience motion created a holographic catalogue that reacted in real-time with customers. 

 

 

Photo: Matthew Carasella/Bloomingdale’s

4. Immersive screens

Microsoft deploys immersive screens in their stores that wrap around the entire retail space. A server synchronizes the images as they flow from screen to screen so the experience isn’t localized to one portion of the floor. The immersive display benefits stores in their versatility. They offer product information, educate viewers on tech topics, provide audio visual entertainment, invite participation through activities like gaming, and update customers on localized events and news.

 

 

 

 

 

 

Photo: Patrick Bingham-Hall

5. Combining high-tech architecture and light

Since the beginning, shopping malls have focused on interior space and retail, often neglecting their exterior appeal. Incorporating interactive public art, technology, light and design into mall exteriors is another effective way of place making and drawing in customers.

The Bugis+ shopping mall in Singapore invested as much into its facade as its interior, and the result is a piece of architecture that is hard to pass by. The building features a curved, crystal-mesh facade. Lighting is integrated into the mesh and controlled with custom software to make it sparkle during the day and glow after sundown. It was designed to involve its surrounding community in an interactive experience; artists and the public can project messages and art into the crystalline architecture on a large-scale. This not only drives mall traffic, but establishes the mall as a cultural and artistic place-maker in the region.

 

 

6. Sound and light shows and simulated experiences

The Mall of America offers a free 9-minute interactive light show every night that focuses on engaging and entertaining younger audiences. The show features lighting that is programmed to a variety of music styles. It can be viewed from various levels of the Mall’s concourse, but those dancing to the music on the ground floor experience spotlights and other lighting effects interacting with their movement in real-time. According to this mom, the multisensory show is effective in driving return traffic, especially for families seeking a unique, emotive experience.

Multisensory experiences are heightened even more intensely at MOA’s FlyOver America. At a small price, you can virtually tour the country’s most iconic landscapes and landmarks in an experience not unlike Brave New World’s feelies, with weather and scents incorporated with sound and visuals.

 

Photos: Active Lab

7. Experiential-only retailers

The Grand Front, a six-storey shopping mall in Osaka, Japan, sets itself apart from most other shopping malls on the planet through interactive technology. You won’t find your typical mall anchor stores at Grand Front—to get a lease there, retail spaces must offer technologically innovative, immersive experiences for consumers. Big brands exhibit concept stores that tackle innovative themes rather than their run-of-the-mill products.

The mall also houses the Innovation Lab, which showcases startup businesses that use mall-goers as guinea pigs to beta-test products like the aforementioned interactive mirrors. Grand Front Osaka also has a Knowledge Capital devoted to edutainment, bridging the gap between retail and multisensory experience. The mall is part of a larger goal not only to resurrect shopping mall longevity but to stimulate a declining tech sector among the city’s largely ageing population.

 

8. Robots

A 4-foot humanoid robot has started spicing up retail spaces around the world. “Pepper” is touted as the world’s first robot that understands and recognizes facial expressions, voice, body language and emotion, and is capable of carrying out basic conversation. The robot is useful in a variety of settings including the home, but introducing the robot to retail space helps welcome and direct foot traffic in conjunction with human staff.

The robot has the added non-human benefits of interactively entertaining kids while their parents are browsing, and following up with customers after retail exchanges. The impact of Pepper on retail space is so promising that the first 1000 Peppers sold within their first minute on the market for about $1600 USD a piece.

Perhaps the most intriguing interactive component to Pepper in a retail space is its ability to style. The robot is programmed to understand how inventories of clothing items fit different body types, and provide detailed personal style advice to individual customers. Too shy to get an opinion from a stranger? Just ask Pepper.

 

If you could describe the future of shopping malls in three words, they would be interactive public spaces. The world’s most thriving shopping centres stay ahead by offering technology and experiences that e-commerce can’t.

It’s not realistic for all shopping malls to adopt these expensive interactive technologies like robots and touch-screen mirrors—but making relatively small installation investments provides opportunities to engage with customers in new ways and create offline social platforms. These will have a huge impact on the success of shopping malls, not only economically, but culturally in their communities.

Any way you slice it, years of overbuilding means that not every mall will survive the coming generations. Those that do will undoubtedly make use of technology, art, and interactivity in their public spaces.

 

To learn more about a Limbic Media product that’s making public spaces interactive, check out Aurora.

Interactive Technology and the Future of Shopping Malls as Public Spaces

 

 Just blocks from Limbic Media in Victoria, BC stands a $72 million construction site that will become a shiny new Mayfair Shopping Centre next fall. It might be hard to find reasoning behind such an investment when the smell of death seems to be in the air with shopping malls. Are shopping malls just another blip on the capitalist timeline, like newspapers or drive-ins? Or are they a cultural necessity that needs to adapt to a changing consumer landscape? An in-depth look at successful shopping malls in today’s market points to the latter—and they’re adapting through technology, interactivity, and place making. This is the first in a series of Limbic Media articles on how shopping malls are changing as public spaces.


Where does shopping mall culture come from?

The role of shopping malls has shifted from generation to generation. Unless you’re riding a roller coaster at the West Edmonton Mall, we now usually think of shopping malls strictly as consumption centres; places we can’t afford to be lured into for the sake of browsing or hanging out, unless we’re in-and-out for something specific. In 2012, venture capitalist Chris Dixon wrote that the future of “offline commerce will serve only two purposes: immediacy (stuff you need right away), and experiences (showroom, fun venues). All other commerce will happen online.” With this changing tide, it’s easy to forget the history behind shopping malls as social placemakers.

 

The earliest shopping malls in North America opened in the 1920s to mirror the automobile industry’s rise. Malls provided easy car-accessible centres for family outings. As suburban invasions of the 1950s moved people away from social hubs in city centres, architects like Victor Gruen, famous for pioneering shopping mall design, saw this cultural shift as an opportunity. Malls could drive consumer traffic by getting people out of their cars and into  commercial spaces conducive to public social interaction in a landscape where there was none. If people had an interactive and engaging place to shop in, they would keep coming back.

 

By the early 1980s, large centres like the West Edmonton Mall usually contained social, non-retail areas like open-air restaurants, skating rinks and even themed amusement parks. Around this time, shopping malls were eating up 50% of retail profits across the United States.

 

Why are shopping malls “dying?”

Flash forward to the post-recession years. Retail outlets like Sears, Macy’s, and Target, the anchor stores of shopping malls, have been filing bankruptcy and closing their doors en masse. When it comes to the demise of shopping malls, in concurrence with Dixon’s prediction, online retail is blamed as the culprit.

 

People are spending more of their dollars online for its convenience and the credibility that online reviews provide. Between 2010 and 2016, Amazon’s sales grew from $16 billion to $80 billion, almost four times what Sears made in 2016. According to the U.S. Department of Commerce, e-commerce accounted for 11.6% of total retail sales that same year, which seems fairly in-line with the number of mall closures—predicted at 15% over the next decade in the United States.

 

Accusing e-commerce exclusively for killing malls is probably an oversimplification. Mall visits declined about 50% in the few years after 2008. With the economic crash, people generally had less disposable income—but their attitudes about consumption, especially among millennials, also changed. People increasingly value multisensory experiences over, or alongside of, the attainment of goods.

 

There’s also the issue of real estate. When anchor stores like Sears and Target close, they leave hundreds of thousands of empty square footage for over-built malls to fill. Finding tenants who are willing to sign expensive leases in a less-than-promising retail market is hard, and malls have to shutter. It might be more accurate to say then that shopping mall deaths are due more directly to the misuse and cost of space rather than a massive move of pedestrian traffic to an online marketplace, at least for the time being.

 

The increasing death of shopping malls seems concomitant with the death of a certain sense of community that malls originally set out to encourage in the 1950s. Inner cities are becoming more unaffordable and more communities are dispersed in urban sprawls. People have less time to join or establish community groups because they are working ever harder to support themselves and their families, and they now have online social platforms to engage in at their convenience. Ironically, the increase of virtual connection has left us feeling no less socially isolated—some would argue even more isolated—than before the rise of social platforms via the internet.

 

 

What are malls doing to reverse the trend?

The new demand for experiential consumption combined with real estate barriers means that shopping malls need to reinvent their spaces if they hope to survive the coming generations. Malls with the same stores and brands on repeat are boring to customers. Centers need to create unique experiences and spaces to attract visitors and keep them returning.

 

One solution is to embrace digital technologies that are seen as a threat to offline shopping and use them to transform shopping malls into multisensory experiences. Malls are reimagining their public spaces into entertainment centres where audiences can participate. By offering this kind of social currency to consumers who value multisensory experiences over physical products, shopping malls increase return foot traffic. Another approach is to add learning components to multisensory displays. Visitors are given opportunities to learn about products while they’re engaging with them, which is especially effective with younger audiences.

 

In the last decade, shopping malls have seen a rising demand for “edutainment.” Edutainment centers combine themed entertainment with experiential learning and high-tech games. 53.8% of all indoor entertainment centers in the world, mainly by the companies Legoland Discovery Centers and KidZania, opened between 2010-2015. This growth reflects the positive effect that combining immersive edutainment and technology with retail have on returning visitors and overall demand for multi-use shopping malls. Because of increased vacancies from anchor store failure, edutainers also don’t have to worry about purchasing land or creating infrastructure for their business—it’s a win-win situation.

 

But shopping malls don’t have to go so far as full-fledged edutainment centers to increase foot traffic. Spaces filled not only with art, but immersive art and displays, are the most likely to attract and keep visitors around. Brands and commercial spaces are catching onto the idea of placemaking by using interactive and immersive art to engage and educate audiences about their products. To create a narrative for the new Ford Fiesta, Ford created a pitch-black maze to create a tactile experience and a 360º mapping projection to virtually immerse viewers in the new model.

 

North American retail outlets are catching up to new immersive technologies to offer shoppers mall experiences that they can’t achieve at home or through e-commerce. Interactive mirrors, navigation touch-screens platforms, robots, augmented reality displays and smartphone apps are being adopted by shopping malls to create an emotive, participatory experience for consumers. Interactive technologies like apps also help malls keep track of foot traffic and find out what’s working.

 

What is the future of shopping malls as public spaces?

Ironically, the future of shopping malls relies on a combination of both emerging and traditional values: creating interactive, engaging experiences through technology, and using them to foster a sense of connection and community that malls seem to have lost since their inception.

 

If malls hope to survive, they need to invest in non-retail spaces that involve people and focus on community-centric marketing and placemaking. Without necessarily investing in massive edutainment ventures or technologies, the return of investing in public art and displays is huge, even if its main purpose is merely to raise the space’s profile and attract foot traffic. Much to the reprieve of shopping malls, the demand for non-retail experiences is still very much alive and well, even if brick-and-mortar retail is on the decline as an exclusive use of space. 

Scroll to top