Limbic Media

Limbic Media

Category: Sound

Adventures in Perception: Aurora

Limbic Media’s Project Technical Lead, Jason Sanche, recently wrote about Aurora in a paper for a university course on Neuroaesthetics. Jason is finishing his Computer Science degree at the University of Victoria and couldn’t have timed this article better with our recent posts about multisensory technologies and their effect on brain development and behavior. The following was adapted from a series of papers exploring perceptual experiences inspired by the artists’ discoveries and insights with specific artworks, and in this case, Aurora.

Aurora QA Station

Aurora: An Exploration in Perception

This article is an exploration of my perceptual and aesthetic experience of Aurora, a software platform developed by Limbic Media to map intricate sound qualities to light. Aurora listens and recognizes subtleties of sound and displays sound as patterns and shapes within two- and three-dimensional matrices of LED light. Aurora elegantly visualizes music with the subtlety of a musician’s ear.

Sounds have incredible texture, depth and emotional resonance, but these facets of sound often go unnoticed. Input from other senses, thoughts, and emotions, especially with the proliferation of screens, continually eclipses our simple awareness of sound. Subtleties get sublimated into the background of our experience. However, the act of listening attentively realigns the mind with time in a  constant, steady, somatic-acoustic present awareness. By showing sound as light, Aurora leverages the domination of visual stimuli and brings attention to sound.

Aurora’s hardware technology is, as Marshall McLuhan would say, an extension of our senses. In the same way, Aurora’s software is an extension of our minds and its neural and perceptual networks. Aurora performs similarly to synaesthesia—a perceptual phenomenon in which stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway. We naturally create images when hearing sounds, which might be an inherited survival trait to anticipate danger from threatening sounds. Incredibly, it also applies to appealing sounds like beautiful music. With our eyes closed and our attention narrowed in on music and the field of mental images, a synaesthetic effect involuntarily transforms sound into mental imagery. Aurora extends this phenomenon into shared space.

The following is a subjective exploration of this idea as a written stream of consciousness about the experience of Aurora reacting to a piece of music:

As the music streams from its digital storage on a cloud of distributed data into the physical network of this room, through my computer, through the mixer and into Aurora which, in real time, processes the signal through digital filters, determined by code, transformed into patterns and colors imitating the harmonies, rhythms and beats, transmuted into bursts of photons varied by a full visible spectrum of color and coordinated patterns with the complexity of the wave patterns in the ocean and captured by my eyes, translated in the optical nerve back into electrical impulses, and again, in real time, perceived as what I believe is sentient to the experience and understood as meaningful. As I write and watch the dancing lights, making the music more beautiful, I perceive and write and know this harmony of embodied sensory experience augmented by technology-as-art.

The study of neuroaesthetics looks at how the mind perceives and attaches meaning to art, beauty, and ugliness, how we fixate on and identify value, and how art produces emotional reactions. A system like Aurora provides a rich and fascinating angle to explore interactivity in neuroaesthetics, and specifically how the perceptual feedback of sound visualization plays into the brain’s implicit synaesthesia.

When sound-to-image happens externally, how does that affect our internal imagination of sounds and music? How important is sound-to-image synaesthesia to our ability to thrive culturally and socially? Can technology like Aurora produce a shared synaesthesia similar to shared public experiences during a film, concert or theatre performance? In participatory public theatre like Sleep No More, the play creates an immersive experience by breaking down divisions between actors and the audience. Can Aurora similarly produce immersive shared participatory musical synaesthetic experiences? The potential is there.

 

Innovation Tree, Victoria, BC

Art’s Role in Imagined Embodiment

Imagined embodiment has been a common theme throughout my explorations in perception. The mind constantly reinterprets its sense of self and embodiment in the world through imagination and dreaming, and the habitual sense of self is usually reinforced if we are unconscious of this process. However, with the right attention and tenacity, we can have full control over our self identity and full freedom from its limiting influence on our inhibitions. Anyone can imagine themselves as anything or anyone, and with enough practice, anyone can act beyond their usual identity. Most people enjoy an occasional respite from the trappings of their identity through events like halloween, masquerade parties, games, and to some extent, books and films that transport us into characters we can safely identify with.

One important role of art is to challenge and disrupt habitual identity through the perceptual experience of imagined embodiment, and made possible by mirror neurons. Conscious engagement in this process through art can introduce viewers to new horizons in self-knowledge.

PGNB Prismo 2017

Interactive Digital Art and Synaesthesia as a Method of Embodiment

The most relevant method of imagined embodiment to Aurora involves the exploration of synaesthesia. Synaesthesia is a unique doorway from the visual to the aural. If we pay attention to sound and its effects on the imagination, it has the potential to create a transformative experience and disrupt habitual sensory perception. Experiencing synaesthesia consciously by meditating on music or sound and absorbing mental imagery restores attentive listening and its meditative benefits. Interactive digital art like Aurora uses technology to leverage synaesthesia and bring audiences back to the present through attentive listening.

Aurora’s lights react to sound the way our mind would visually imagine the source of any sound. The nature of sound and the act of listening have a unique quality that visual perception does not—sound disappears nearly as soon as it is heard; it is more ephemeral and decays quickly through friction unlike most visual objects, which tend to persist until they are destroyed, or decay over longer periods. Because sound does not persist very long, attention to sound created a synchronization to the ineffable flow of time: the steady, consistent arising and dissolving of the soundscape. By perceiving sound as light we tune into the act of listening which is an important way of staying balanced and present in a sensorially fractured world. This allows the mind to be present in time, which is its natural state.

 

For updates on new content, sign up for our newsletter.

Heads-up vs. Heads-down Technology: Impacts On Brain Development

Nicholas Carr

Photo: Antenna

When it comes to brain development, technology gets a bad rap. Nicholas Carr, author of The Shallows: What the Internet is Doing to Our Brains, compares our new ways of interacting to a backwards evolution: “We are evolving from cultivators of personal knowledge into hunters and gatherers in the electronic data forest. In the process, we seem fated to sacrifice much of what makes our minds so interesting.”

Last week’s blog tailed off with a different comparison. We suggested the connection between heads-up technology, which forms genuine social connections, and how those technologies reflect early childhood experiences. This week delves more into the question of how technology impacts brain development, especially when parsed by what we like to call heads-up and heads-down varieties. This is pertinent among children who are now spending their formative years engaged with technology. Technology observably impacts our social habits, but more easily overlooked is how it physically rewires our brain.

 

Child Brain Development

Photo: The News International

How is technology “rewiring” our brains?

The speed at which technology is changing and entering our world means that research on how it psychologically influences us can’t possibly keep up. But one thing is likely—technology is rewiring our (and especially our children’s) brains in ways that we haven’t yet encountered and don’t fully understand.

A 2007 UCLA study measured brain activity of experienced vs. non-experienced web-users in their prefrontal cortexes, areas associated with problem-solving and decision-making. The study found localized brain activity in experienced users much higher than their counterparts, even though brain activity was comparable across all participants when exposed to non-internet-based reading tasks. After instructing the non-users to engage in internet use for one hour per day for six days, the study was repeated and found comparable brain activity across all users.

Even though this is only a single study in a large body of research on technology and brain activity, it just goes to show the incredible plasticity of our brains and how quickly technology exposure has a physical and yet subconscious influence on our social and sensory experiences. It’s also worth noting whether studies of this type differentiate what we’d consider heads-up or heads-down technology; this particular study on internet use would definitely fall under the latter type.

That isn’t to say that internet exposure, or technology in general, is necessarily rewiring our brains in bad ways, just different ways. Using technologies like the internet has a tendency to frequently redirect our attention, forcing our brain to spend energy reorienting itself at the expense of comprehension. These sacrifices are known to researchers as switching costs. Considering the overload of advertising, hyperlinks, and other visual re-directors, it’s no surprise that humans are cognitively paying a higher price for switching costs than ever before.

 

The pros and cons of heads-down technology on brain development

If the influence of technology burdens us with switching costs, what are the benefits of technology, especially of the heads-down variety, in rewiring our brains? Nicholas Carr, a fan of Neolithic metaphors, describes the cognitive skills brought on by internet consumption as “primitive mental functions:”

  • Hand-eye coordination
  • Reflex response
  • Visual-cue processing
  • Fast-paced problem solving
  • Credibility assessment
  • Pattern detection

While these influences have their benefits, they are seemingly more base than socially interactive in their enhancement of human brain function. Evidently, a balance of the two is ideal for healthy brain development, especially in children. Patricia Greenfield, a developmental psychologist at UCLA reviewed over 40 studies in 2009 to assess the effects of technology and media on intelligence and learning. According to Greenfield, the “widespread and sophisticated development of visual-spatial skills” has come at the expense of “deep processing,” “mindful knowledge acquisition, inductive analysis, critical thinking, imagination, and reflection”—skills that, one could argue, are more relevant to forming our social tendencies early on in life.

 

Balancing technology and brain development

We’ve already discussed how heads-up technology is designed to encourage face-to-face interaction in social settings. It engages users with their surrounding environment rather than isolating their attention spans. In terms of technology rewiring our brains, there are parallels between heads-down technology and the visual-spatial skills it enhances, and heads-up technology and the “deep processing” skills Greenfield speaks of.

In an ideal world, humans would have the self-awareness and control to use both technology types in healthy amounts and encourage the same use among their children. “Heads-up technology” is only a novel idea because it is usually the exception rather than the norm, however. Heads-down tech just seems to be more widely accessible and leaves us more susceptible to unhealthy, addictive tendencies. If we hope to avoid the detrimental effects this has on our (and our kids’) brain development, there needs to be more research on what separates heads-up from heads-down technology and how the use of each is “good” for brain development.

 

Photo: Super Glam Moms

Multisensory technology and brain development

One area of research that studies the influence of what we’d consider heads-up technology on brain development focuses on children with Autism Spectrum Disorder and multisensory experiences. One such study, conducted at the Albert Einstein College of Medicine at Yeshiva University, found that children diagnosed with ASD do not process multisensory information as effectively or quickly as other children in that age group. This kind of research points to why there is a burgeoning industry of multisensory experiences and therapies geared toward families living with autism. The Children’s Museum of Atlanta, for example, hosts a regular multisensory program. One of the Yeshiva study authors went as far as to compare multisensory integration therapies to a “military-industrial complex.”

While there is definitely more room for research on the direct impacts of multisensory and heads-up technology on our brains, perhaps the approach’s popularity has more to do with the impacts of heads-up technology regardless of a diagnosis like ASD. Multisensory tech encourages our brains to fire on all cylinders rather than tune out certain aspects of brain development, and could be good for encouraging critical thinking and focusing attention in young brains that respond well to non-conventional learning styles. Continued research in this area will help us decide how to balance our exposure to various kinds of technology and approach product design, especially for children.

 

This week’s article evolved into what feels more like a research paper than your average blog, closing with more questions than solutions and answers. We know relatively little about the long-term effects that technology consumption has on the human brain, and we know even less about those effects when you try separating technology into categories like heads-up and heads-down. At the end of the day, a healthy dose of both types is likely ideal for well-rounded brain development, and we’re seeing a growing emphasis on engaging, multisensory technologies to influence that development, especially among children and those with disorders like autism.

One way of delving further into this topic is by looking more specifically into the science behind multisensory experiences. Public spaces are investing more into services and displays that captivate and engage audiences on auditory, visual, tactile, and even olfactory levels. Tune in next week to find out more about what’s behind the growing appeal for multisensory technologies.

 

Don’t forget to sign up for our newsletter for more great content!

 

Santa slays: Why holiday shopping mall displays are worth investing in

As cherry trees begin blossoming along Victoria’s boulevards, thoughts of Christmas planning are at least several months down the road for the average person. However, big-time holiday event planners like shopping malls and other public centers are already getting the ball rolling for the 2018 season. Our previous article discussed how shopping malls are adapting their public spaces to survive coming generations. Malls are finding themselves in a consumer landscape that is spending more on multisensory experiences and online shopping, and they must adapt their public spaces accordingly. Christmas is the most lucrative time of the year to invest in that change, and the easiest way to do it is through interactive holiday displays. This is the second in our series on how shopping malls are changing as public spaces.

 

Santa reading his list Offline displays offer something that e-commerce can’t 

As more consumers move to e-commerce, malls are making or breaking their success by transforming their public spaces into immersive art displays and edutainment centers that incorporate participatory technology. Since the recession in 2008, consumers have less disposable income but are also more inclined to spend experientially. Now that the millennial generation is becoming parents, they are channeling those experiential values through their kids as well. Shopping malls are waking up to the fact that offering interactive components to their space not only attracts customers, but keeps them returning. These displays offer something consumers can’t get by shopping online, and they’re willing to take the trip to brick-and-mortar stores to get the experience.

Shopping online is seen as a big threat to offline material consumption. For the time being, however, recent numbers show that only 11.6% of retail sales occur in an online marketplace—and all it takes is a short visit to shopping malls in early December to see that there is still a huge demand for over the holidays. Investing in non-retail, interactive public displays provides a huge opportunity for shopping malls to raise their profile not only during the season, but for the rest of the year.

While adapting space to meet these needs is worth the year-round investment, shopping malls create displays during the Christmas season almost by default—so standing out and creating something that other public or offline spaces aren’t is critical to stay afloat. You just can’t get a spot on Santa’s lap over the web.

 

Christmas shopping mall

It’s a win-win for consumers and non-retail business

It’s no surprise that shopping malls make a killing over the holidays. However, a deeper look into the holiday display industry reveals a seasonal employment sector that is surprisingly lucrative. When you picture the 5000 mall Santas that are estimated to be employed across the United States every holiday season, it’s the stuff of an obscure holiday-themed VICE documentary on American subcultures. Mall Santas rake in anything from $10,000-$60,000 USD over the five to six-week holiday season, some even earning $500 USD per hour.  

And it’s not just Santas that benefit from seasonal employment. There are at least two professional santa claus schools in the United States—one in Denver, Colorado, and the “Harvard” of Santa Schools, employing faculties that have been pumping out an annual class of bearded graduates as early as 1937. Behind every holiday mall display is also a team of event planners, photographers, lighting installers, set designers and assistants to attend the millions of Santa-display customers every year.

Santa mall displays are becoming more sophisticated and interactive than ever before. The aforementioned Santa schools include workshops on the latest gadgets to get Santas familiarized with the kind of edutainment kids are after, and to offer an extra level of interactivity on site. Phone alerts allow customers to avoid lineups or queue specifically for a black, white, asian, or ASL-speaking Santa. Some bigger malls even have themed interactive Frozen or Shrek displays to occupy customers while they wait. Holiday displays are more engaging for visitors and as a result, increasingly profitable for their organizers.

 

Counting money The return of investment is huge

The ROI of having an engaging holiday-themed display at a shopping mall is staggering. One mall can earn up to $1-million per season directly from a single Santa display. That doesn’t include the retail revenue from visitors who shopped while they came in to see the display. The trade group International Council of Shopping Centers found that 70% of shoppers made purchases while they were at the mall specifically to see Santa. If an interactive display has the potential to draw that kind of return for six weeks of the year, it makes sense that shopping malls are investing more in non-seasonal immersive art and participatory displays.

Monetary returns on investing in displays are convincing, but there’s also huge value in intangible returns that is overlooked, especially when aesthetics are often perceived as secondary to profits.

A mixed-use building in LA’s Koreatown neighbourhood, housing residential and retail space over a busy subway, invested $75,000—or 0.06% of the development’s total cost—into a large-scale murals on the building’s facade. The public art ended up getting featured on the cover of LA Times’ news and culture section, not only marketing the display but also giving the developer invaluable international recognition as a cultural influencer. The display ended up in dozens of publications, became a popular location for photo and video shoot and news backdrops, and served as a marketing image for the transit authority.

Similarly, the California Department of Transportation developed an HQ in a nearby neighbourhood. The development required 1% of its total cost to be dedicated to public art, as mandated by the State of California. The result was an installation of architectural neon and argon light tubes that emulate car tail lights in motion. The public art was a big hit. The Department receives a regular income from photographers and filmmakers who use the space as a site (there are definite Bladerunner vibes), and the architectural light installation has gained a global reputation in the public art world.

While these examples aren’t holiday-themed (displays can usually be adapted quite easily, however), they confirm that the ROI for non-retail investments in public space are well worth it, both fiscally and intangibly—and these public art examples aren’t even interactive! When displays, holiday-themed or otherwise, are raised to a participatory level that the public can actually engage with, that value is elevated even moreso. Not only does it benefit those commissioning and creating the art, but it also fosters a sense of community and placemaking for those in direct contact with it.

 

Present addressed from SantaThe takeaways

In an uncertain offline retail market, some shopping malls might balk at the idea of investing a substantial sum of money into holiday displays. However, in line with Henry Ford’s saying that “a man who stops advertising to save money is like a man who stops a clock to save time,” not investing in the impact of a public space is sacrificing the very element of shopping malls that is key to their survival.

Holiday displays like Santas that offer a mixture of public art, interactivity and technology helps malls stand out from other public spaces, profit the most from retail and employ more people seasonally. Because holiday displays also have the potential to raise a shopping mall’s profile, investments into public art, particularly interactive public art, are well worth the return if they can also adapt to non-seasonal audiences that value multisensory experiences outside the home.

Subscribe to our newsletter for more great content like this!

Scroll to top