To some, reality requires the presence of something in a physical, tangible state.
We often have trouble believing what we see if we can’t touch it. If we actually can’t see it, we are likely in a battle between the possible and the tangible. Our level of awareness and presence can either heighten or constrain our ability to see and interact with those things not observable in the physical, tangible world. There are infinite examples of things existing without our ability to validate them through sight, sound, smell, touch or taste. X-rays, gamma rays, any sound below 20Hz, any light within the ultraviolet spectrum, quarks, neutrinos and other subatomic particles are examples of existence that humans cannot observe without external aid.
Technology has a wonderful power to shed new light, challenge existing frameworks and change the way we interact with the world. The complexity of the technology and whether it actually satisfies a specific human need are factors that lead to its adoption.
Is technology’s greatest challenge to be mindful of its feedback loop to humanity?
What makes something real?
Is it something happening in the physical, tangible world?
Or something that allows our minds to experience it as it would occur in that world?
The electrical signals emanating from the gray matter locked in our skulls are far more capable that our current understanding, and the technological architecture of the future is closely coupled with the human brain.
I recently was able to spend time with some of the world’s leading innovators in Augmented and Virtual Reality at an event called FutureXLive, which was thoughtfully curated by the digital creative agency, Moxie.
The Woodruff Arts Center provided an inspiring backdrop for this unique exchange of ideas surrounding the following topics:
- Latest technical innovations in Augmented and Virtual Reality
- Research studies on empathy and telepathy
- The age old art of storytelling
- The foundational principles of technology adoption
- Brain machine interfaces
- How the invention of a platform does not create a medium
- Meaningful application of technology for the betterment of the human condition
- Sneak peeks into a future where we beam our consciousness from station to station across the universe described by a theoretical physicist credited for the development of string theory
It was eye-opening and thought-provoking, leaving me with guarded optimism and an appreciation for the balance between humanity and technology. I thought I would share some of the highlights from my experience with the great minds at FutureXLive.
I’ll start with some quick definitions.
Augmented Reality – Supplemental or blended, adds to or enhances the existing world
Virtual Reality – Creates a new world entirely, fully immersed
The following is designed to be a short-form journey and virtual sprint into these worlds from the minds of its leading experts. While each session explored various topics, there was a sense of understood responsibility to enhance and enable without replacing or eliminating, which should be shared between both the innovators and users of those technologies.
Podcast and Video Host at HOWSTUFFWORKS
On the state of the gear …
Some of the issues with adoption of AR and VR is the gap between the expectations and the reality of the current technical delivery. Jonathan presented a state of the industry related to the gear behind the experience. Google, Sony, Microsoft and others are delivering VR equipment to support the various environments (head space, desk space, room space and world space). These technologies are focusing on gesture control, voice tracking and eye tracking functionality.
On cutting the cords …
Intel launched Project Alloy to develop an untethered version of a VR headset, which was recently abandoned largely due to a lack of interest from a partner to build them. They are currently focusing on high speed networking that will hopefully enable others to build untethered access to VR environments.
Interesting uses of AR – Franklin Institute (Terracotta warriors)
VR and AR could act as time machines if applied to historical experiences that would be otherwise impossible to see. Imagine full immersion into the D-Day invasion, or being placed in the middle of Bob Marley’s One Love concert, or sitting on the grassy knoll before JFK’s assassination.
Jonathan presented an excellent holistic view on what AR and VR can do …
These new technologies provide for the extension of our human abilities.
Dr. Helen Papagiannis
Author of Augmented Human: How Technology is Shaping the New Reality
The core themes of her talk centered around the concept of balancing humanity with technological innovation. With this technology, we can extend our abilities, rather than replace our humanity. Dr. Papagiannis took us on a journey through the senses and how technology innovations can extend our abilities and even provide superpowers.
Caveat – I haven’t used these devices personally, so I am not providing a technical review or recommendation. They do seem really interesting, and I’d love to check them out.
Orcam developed MyEye, which is a wearable technology that turns visual information into audio information providing independence for the visually impaired.
Imagine you are sitting in a restaurant, frustrated by the fact that you can’t have a conversation due to the amount of noise. What if you could control your audio environment like an audio engineer in a studio? What if you could bring levels of independent sources up or down, adjust EQ and add effects? Doppler Labs created Hear One (in ear headphones + app) that allows you to personalize your audio environment wherever you are.
Some of the biggest challenges with the adoption of VR and AR stem from haptics or tactical feedback. We can see things that aren’t really there, but we haven’t been able to feel them. Haptics involves the science of applying tactile sensation to VR/AR experiences to create the illusion of felt sensations. Disney Research has created a new wearable technology to bring the idea of touch to the forefront. Revel uses weak electrical signals in varied amplitudes and frequencies to deliver tactile experiences. The signals are generated through a device that can be placed in your shoe, on a chair, or in a touch screen.
What if you could add all the sugar and salt onto your food without physiological consequence? While digital knives and forks are not currently in production, Dr. Adrian Cheok is experimenting with a Digital Taste Interface that stimulates taste sensations directly on the tongue. Through current, frequency and temperature, he has been able to create taste experiences ranging from sour to salty.
Dr. Papagiannis closed her talk with the idea that these experiences should be built and delivered to enhance and enrich rather than replace humanity.
I look forward to exploring these ideas further in her book, Augmented Human.
Dr. Adrianne Randolph
Associate Professor, Kennesaw State University and Executive Director of BrainLab
Dr. Randolph discussed the idea of telepathy and the ability to use the signals of the brain to communicate. At her lab and incubator, BrainLab, her team uses non-invasive brain computer interfaces to study computer technology and the human neurological system. Her definition of hands free communication is controlling computer devices with nothing more than your thoughts. With an electrocap and a few electrodes, her team is able to record the electrical activity of your brain. This information is translated into a language that a computer can understand, which will ultimately allow your brain to control a device.
But, let’s take this one step further …
We already implant foreign objects into our bodies to enhance our experiences.
Pacemakers, hip joints, insulin pumps, artificial lungs …
What about the idea of the brain pacemaker?
Companies like Neuralink (Elon Musk), Kernel and Neurable are working on a brain machine interface that is actually implanted in the brain. The idea of neural lace was born out of The Culture series of science fiction books from Iain M. Banks.
It sounds like we may be dropping the fiction part.
Other news in brain research …
Researchers at MIT have figured out how to implant memories into the brains of rats.
A team at Wake Forest discovered a way to erase and restore memories in rats.
Future technological innovation will be closely coupled with the human brain, and these fascinating discoveries have the ability to wreak havoc or improve lives.
Dr. Janet Murray
Professor of Digital Media at Georgia Tech, author of Hamlet on The Holodeck: The Future of Narrative in Cyberspace
Her talk balanced skepticism and optimism related to VR/AR and the long-standing and proven technology of storytelling. Similar to the camera being a platform supporting the medium of filmmaking, “barely inventing the platform (VR/AR) is not inventing the medium”. I digest that comment as while the technology is available to create content on the VR/AR platform, we haven’t quite figured out how to effectively tell the story and create the medium to drive adoption.
According to Dr. Murray, a successful VR experience will “shape the sense of agency”.
She shared an amazing installation that allows the control of an animated ocean environment through the brainwaves of a single participant, which are picked up though an EEG headset device. By relaxing and concentrating, the participant can control the ocean environment.
You Are The Ocean
Founder and Creative Director, Trick 3D
Chad defined AR and VR as the “real world plus superpowers”.
He shared a story about a wonderful project he produced for the Make a Wish Foundation. A boy named Zayden had a wish to travel to Saturn in a red rocket ship, so Chad and his team let Zayden act as the creative director for his own personal experience into space.
If this video doesn’t make your heart sing, I don’t know what will.
On mapping our world to build new worlds … He also mentioned that the next few iterations of smartphones will give us the ability to create 3D maps of the world, which will give us the basis to create new versions of our existing world.
As our technological capabilities continue to exponentially increase, we will have to find ways to balance how we use this particular superpower.
I liked Chad’s solution.
When we are presented with endless capabilities we must always “choose beauty“.
VR Creator and Experimental Filmmaker
Gabriella’s last project, Traces, focused on using VR as a tool for memory through the exploration of Alzheimer’s disease. The film was based on a series of audio interviews that Arp conducted with 88 year old Willie White, who had been living with dementia. These interviews turned into VR experiences of major events in White’s life at or near the actual event locations in Mason, TN.
She also shared my new favorite quote on memory.
“A memory is not an object preserved in the museum of our minds. It is a living, changeable thing that is shaped by who we are when we encode it, and who we are when we retrieve it.” – Anne Basting
Writer, Creator and Designer, META Studios
Paul took the audience on an entertaining journey through traditional storytelling, how audiences consume content and technology adoption. VR storytelling has struggled within the confines of the traditional entertainment industry because it doesn’t really fit in any one department. It sounds like it will get there, but it will be driven by how and when the consumers embrace the platform. Paul presented a historical account of technological adoption beginning with the Luddites and weaving equipment and concluded that “adoption happens when it provides convenience and help for the influencers of society”.
For a deeper look into technology adoption, he recommended Calestous Juma’s book Innovation and its Enemies: Why People Resist New Technologies.
Director of Rehabilitation Innovation, Mt. Sinai Health System
David is a physical therapist with a PhD in Neuroscience, and he applies innovative technology to Mt Sinai’s standard of care.
On the brain and high performance athletics …
As a CrossFit coach, I’ve always been a huge proponent of our body’s ability to outperform the expectations of our mind. Putrino participated in a study that scientifically proved there is often more in the tank. Along with Red Bull’s High Performance Division, he studied the role the brain plays in the fatigue process. By monitoring the electrical signals of the brain, they were able to understand the delta between when the brain quits and the actual biological reserve available.
On applying new technology in unique ways …
David presented another inspiring moment similar to Trick 3D’s Zayden’s Wish Project. A technology that he and the team from Not Impossible Labs developed was reapplied to help Joe Scianni, a jazz pianist and composer stricken with Parkinson’s disease, play music again. The original system was developed to allow the deaf and hard of hearing to experience live music by translating it into vibrations that activate a wearable set of devices. The team put one of the devices on Joe, and the vibration technology counteracted the tremors from Parkinson’s disease.
What a beautiful application of how technology innovation is changing lives.
For more on Joe’s story, check out this short film from Not Impossible Labs:
Dr. Michio Kaku
Theoretical Physicist, Author and Professor
Dr. Kaku presented some mind-blowing ideas in a simple manner coupled with some well-timed humor.
On the challenges with VR …
There are “differences between what you see and what you feel, then you throw up”.
Audio and haptic feedback continue to be the barriers for believing the VR experiences.
On learning …
“Concepts and principles, not memorization”
On consciousness as a series of feedback loops …
“Well, I have a new theory of consciousness, and that is: Consciousness is all the feedback loops necessary to create a model of yourself in space, in relationship to others, and in time, especially forward in time. This means that animals are conscious, and we can even rank them numerically by counting the number of feedback loops involved in each of these behaviors. So a thermostat would have one unit of consciousness that measures temperature. A flower would have maybe 10 units of consciousness because it measures temperature, sunlight, gravity, moisture, things like that. A reptile would have even more, maybe several hundred, because it locates its position in space. Then monkeys are even higher than that because they have to locate their ranking in society via emotions.”
According to Dr. Kaku, the level of consciousness is measured by the number and complexity of the feedback loops. Humans are different because of our self-awareness and our obsession with time. Thanks to the addition of our pre-frontal cortex, our reptilian and limbic brains are balanced with our ability to understand time and our position within it. Animals have no perception of time.
Dr. Kaku challenges, “Try to teach your dog the meaning of tomorrow”.
On digitizing consciousness …
As we continue to map the brain through the Human Connectome Project, we will construct a more detailed understanding of the inner workings of the gray matter residing within our skulls. The brain generates electrical signals that travel through a complicated network of neurons to deliver the experience we call consciousness. Through this understanding, Dr. Kaku presented the idea of harnessing consciousness in a technological capacity.
So, how do you digitize consciousness?
Even though it’s beyond the current technology, its coming.
An utterly simplified and paraphrased version of Dr. Kaku’s idea …
Step #1 – Create the perfect robot body.
Step #2 – Duplicate neurons into transistors.
Step #3 – Transistors inserted into robot body.
Step #4 – Move signals from the brain to the transistor-based brain in the robot body
Step #5 – Transfer the experience of consciousness.
Step #6 – Beam your consciousness across the universe from station to station.
Ok … let’s pause and bring it back a bit ….
At the end of the programs, I found myself in an inspired, slightly confused and marginally frightened state. The good news is that inspiration was the leading emotional response, and growth comes from being slightly uncomfortable.
I’m thankful for the experience presented by Moxie as it validated some of Tunewelders’ developmental concepts and projects, it sparked new research rabbit holes , it presented new perspectives on old ideas, and confirmed the importance of balance.
Balance in the combination of technology and humanity.
Balance between storytelling and production-based innovation.
Balance in fear and discovery.
Balance between the possibilities and responsibility.