How virtual reality is increasing educational inclusiveness
By Academy Xi
Share on facebook
Share on linkedin
Share on twitter
February 4, 2021
For over 60 years, Virtual Reality (VR) has shifted the way we experience the real and virtual world, by transforming people’s perspectives and social interactions. As an increasing number of VR companies and products emerge, Equal Reality, a VR company led by Brennan Hatton, is increasing equality by allowing users to look at things from the point of view of those from a different gender, race, or ability level.
According to Brennan, his company enables “Human to human interaction, but on a global scale, and on the other side of the world.”
From NASA to the NFL, VR is also finding its greater purpose in the education space. NASA is now using VR as a way to train astronauts to perform repairs and experiments in zero gravity. The NFL is also using virtual spaces to not only train its athletes but to assist them with workplace issues like discrimination.
Today, an increasing number of industries are using VR as a method of employee training. According to a report by the National Training Laboratory, retention rates for lecture-style learning were at 5%, with reading rates at 10%. Meanwhile, the teaching method of VR scored a retention rate of 75%.
Within the Deaf community, early intervention and education programs play a vital role in ensuring young people have choice and access to appropriate support. Former Academy Xi VR Design student, Nikhil Bora, is looking to use VR to increase the inclusiveness of education within the deaf community.
“To ensure that there is no difference in life outcomes for deaf children when compared to their hearing peers, then the importance of early intervention and education through technology such as VR becomes self-evident,” says Nikhil.
Both Brennan and Nikhil believe the retention rates of VR are high with those that are disadvantaged because of its immersive experience. “It’s about the experience, which is one of the things that makes Virtual Reality so powerful. This is one of the core values of Virtual Reality,” says Brennan.
“You can give people experiences and communicate experiences that were otherwise impossible. Before, the best we could do was describe those experiences, but now, with VR we can put people in these experiences,” says Brennan.
After seeing the impact of VR in education, Brennan joined forces with former colleagues Rick Martin and Annie Harper, and this was the birth of Equality Reality. Equal Reality uses VR technology to create diversity and inclusion training for corporates, startups, and schools.
“We put you in the perspective of minorities in the workplace to experience social interactions from their point of view,” explains Brennan.
The social scenarios include experiences around gender and disability. For example, the disability scenario creates an experience based on the assumptions people make on another person’s state of physical, mental, social or wellbeing. With the gender scenario, the experience is based around the assumptions of roles and a person’s capabilities based on their gender.
Many of the social situations Equal Reality is working on can be often overlooked by the person interpreting the situation. However, with the help of VR, Brennan says, “When you can feel it, and someone is receiving that impact, you can better understand the negative impacts.”
For Nikhil, he started a passion project using VR and believes that “this project will give the opportunity for everyone to learn and explore about deaf awareness, sign language, and how to communicate with deaf people. This will enable the deaf community to have the same equal experience as everyone else.”
While working on the diversity and inclusion training experiences, both Brennan and Annie are also giving up their time to teach children about the importance of VR and how they can start working in it.
“Educating youth about VR is incredibly important because, in the future, VR may very well be in their day to day lives. It will play a large part of how they interact with the world and each other,” says Brennan.
Like Brennan and Nikhil, explore how you can drive positive social impact through emerging technologies such as Mixed Reality, Virtual Reality, or Augmented Reality. Learn more about our online courses in Australia.
Google’s Tilt Brush: A new artistic medium, or something more?
By Academy Xi
Share on facebook
Share on linkedin
Share on twitter
September 24, 2020
At its core, Tilt Brush is a new way of visualising and creating.
Technology and innovation have long been shaping art. A sparkle of innovation, the humble paint tube, brought impressionist painters outdoors – en plein air; if such a shift didn’t take place we’d be without masterpieces like Monet’s Garden (which you can probably visit in VR, by the way).
We then saw the invention of early forms of photography, which influenced art away from realistic representation. In the latter half of the 20th century, iterations of digital imaging software have paved the way for entirely new digital mediums.
Fast-forward to the present day and we have this entirely new, and incredibly exciting, marriage between art and Virtual Reality tech. You can now draw sweeping marks through 3D space; walking into, and around, your own floating brush strokes. Describing the experience doesn’t really do it justice; painting in space is unlike anything else.
Jane Venegas Sproule, Academy Xi’s VR Course Curator, identified a recurring pattern of use; newcomers to Tilt Brush often follow the same routine. The first use is playful, trying out new brushes and creating unintelligible squiggles. The second try becomes more sophisticated, but is still stuck in 2D; students draw in one plane, as if on a big, floating piece of paper. Jane explains, “by (at least) the 3rd try, students begin to better understand and effectively use space and a 3rd dimension.” Having only tried Tilt Brush once, I’m still at the squiggle stage. This transition into 3D can take some getting used to, Jane points out:
“It’s understandably difficult to get used to an extra dimension; the Screen has dominated our lives since the invention of the TV. We are all very accustomed to smartphones and laptops”
Taking a step back from the wonder of it all, you can start thinking about the implications Tilt Brush could have on the way we work, live and play – replacing our 2D digital landscape with this new 3D interaction. Companies are beginning to play with this idea of a Virtual Workspace – but there’s still a lot of room for development.
Tilt Brush was released in April this year. Now on its 7th upgrade, Tilt Brush’s newest features nicely sum up how VR is progressing. You can now work in teams, import and export objects, resize your virtual space, and virtually swap controllers. VR creators are allowing users to transcend physical limitations – allowing us to play with scale and space. “What started as a playful VR experience, is becoming a functional tool”, points out Jane, “the applications for prototyping, collaboration and idea visualisation are endless.” Imagine virtually collaborating on a prototype, with team members from across the globe.
Virtual Reality (VR) and Augmented Reality (AR) are rapidly changing the way in which we perceive the world around us. Mixed Reality is set to be even more immersive and will likely define our digital experiences in the future. Find out what Mixed reality is, how it works and how it’s already being used in everyday life.
While the experiences offered by Virtual Reality and Augmented Reality are informational and consumed on screens, Mixed Reality promises to be far more immersive.
Instead of bringing more of the physical world into our digital screens, technology is primed to use Mixed Reality to take digital data and fuse it with our actual environment.
What is Mixed Reality?
Mixed Reality (MR) is a combination of multiple advanced technologies, primarily Virtual Reality and Augmented Reality.
To understand Mixed Reality, let’s differentiate VR and AR:
Using VR devices, the user’s current reality is replaced within the virtual world. It doesn’t account for the user’s immediate surroundings, thus it is usually recommended to use VR technology in spacious environments where the user won’t accidentally bump into something in the physical space. By design, it isolates the user from their context in order to provide an immersive experience. Current examples of VR devices in the market are the Oculus Rift and HTC Vive.
this technology usually just roughly overlays digital content on top of the real world. It doesn’t account for your surroundings, nor interacts with your space. Simply AR rests on top of any surface, with the physical world acting as a static background for it. Examples of AR devices recently developed were the Google Glasses and Snapchat’s Spectacles.
Unlike AR that simply overlays digital elements on the physical space without considering its unique and changing composition, MR devices constantly gather information about the surroundings. This information will then be used in order to seamlessly place digital content and information on the physical space and allows the user to interact with it.
Unlike VR, you don’t disappear into the digital world, the digital world goes to you.
Mixed Reality may sound like brand new, futuristic technology that is still decades away from being a reality. In fact, MR has been around since the 1990s as a term coined to explore the potential of combining both virtual and augmented reality together.
Even augmented and virtual reality are not brand new, despite their commercial use and applications as of recent. In fact, both VR and AR technology can be traced back to the 1960s.
Today, Mixed Reality is a technology that is quickly making its way into mainstream use. Microsoft’s Hololens is a well-known example of an existing, commercially available Mixed Reality device. It is a holographic computer you wear around your head, with lenses over your eyes that project holograms you can manipulate and interact with as though they existed in your physical surroundings.
It features 3 sensors and 5 cameras that constantly learn your 3D surroundings, and can even remember where you place digital elements in your physical space so that when you use the system again, your windows and apps will be right where you left them.
While Hololens operates on the Windows system and offers applications and software you can interact with as though you would on a touch screen, Meta 2 is an MR device that allows users to interact with virtual objects as if they were physical objects.
These two are just examples of MR technology that currently exists, and gives us insight on what the technology can do now, and what it can potentially do in the future.
How does mixed reality work?
Imagine sitting in an office, where you can see all the material objects that surround you, like walls, desks and chairs, but you can also interact with digital content, like a shared document which updates to the cloud, or a prototype that you can experiment with using a digital interface. These are the possibilities of mixed reality. How does it work?
Cutting-edge technology is able to create mixed reality by:
Using input devices – Mixed reality software takes its commands from a series of input devices which collect data in the real world. Input devices include orientation trackers, magnetic trackers, optical trackers, eye tracking devices and ultrasonic trackers, to name just a few.
Detecting physical objects – Mixed reality relies on object recognition, also known as object detection. This term refers to the ability of the tech to identify the form, shape and position of different objects in the space captured by a device’s camera.
Real environment perception – Mixed reality software uses input devices to sense the environment around it. It maps the space and then superimposes information onto that space in a way that merges virtual and real-world content.
Mixed Reality in the real world
The possibilities and implications of Mixed Reality’s continuous growth demonstrates that the future of MR is both realistic and limitless.
While we can only predict where MR can take us next, its benefits and uses in various industries today and in the near future are astounding:
Mixed Reality in Healthcare and medicine
Mixed Reality simulations are already helping medical students understand their patients better, particularly those who are hearing and visually-impaired. Through MR technology, how these patients experience the real world is simulated, leading to greater empathy and understanding from medical practitioners.
Doctors — and surgeons, in particular — are able to learn complicated surgeries and develop cutting-edge procedures. They can run scenarios and simulate the same conditions in the operating room, allowing them to plan and predict outcomes.
Through MR, first responders can also better prepare for work scenarios safely and without risk, making them better prepared for these stressful situations. MR also helps PTSD patients through controlled exposure in a safe environment and at their own pace.
Mixed Reality in Education
Experiential education is one of the most effective learning and teaching tools. Through Mixed Reality, students are able to interact with what they’re learning unlike ever before. It’s not just visual, audio, or traditional learning methods that will enable students to learn, but actual experiences that promote deeper, immersive learning.
Mixed Reality may also be used in the future to expand on Virtual Reality’s work with immersing people in different cultures, raising awareness to causes in a way that bolsters greater empathy. People will be able to learn about other cultures, social, political, and economic concerns in near first-hand experience through Mixed Reality.
Mixed Reality in Gaming
There is great potential for Mixed Reality in gaming, unlocking an experience that combines the intense and impressive worlds in video games into the actual environment. It has the potential to gamify fitness, as Augmented Reality did with the Pokemon Go phenomenon.
With digital elements being embedded in the actual environment, people will be motivated to physically move to access digital content and experience them hands-free, instead of simply looking at screens on handheld devices.
Mixed Reality in Retail and business
Many possibilities await businesses when they make use of Mixed Reality technologies. Companies are now maximising VR and AR, like providing augmented reality maps to customers for better understanding and access to their stores.
With MR, stores can give customers unprecedented information as they walk through the aisles, empowering customers to make informed and confident purchases without the need for assistance. They can also experience a product or a service before purchasing.
Mixed Reality will unlock a whole new way for people to experience the world, which could mean new products and goods for companies to develop and offer. Just like today’s trend of microtransactions and in-app purchases, businesses can sell virtual wares to populate this mixed reality world.
Examples of Augmented Reality in Everyday Life
Augmented Reality is used in all kinds of everyday apps and software to give us all more immersive and creative experiences. Some of the most widely used examples of everyday AR include:
Snapchat – The filters that the Snapchat app offers use AR to overlay digital content on the user’s face during a video call.
Photo editing software – Even amateur photographers can add artistic special effects to their photos with AR. As an example, Tipit has launched an automated photo editing mobile app that uses AR called Model.
Google ARCore – Google’s platform for building augmented reality experiences uses different APIs and enables your phone to sense its environment, understand the world and interact with information. Some of the APIs are available across Android and iOS, allowing you to share AR experiences.
Interior design apps – Taking the guesswork out of redesigning a room, the latest interior design apps can use AR to show you what a space will look like with a certain choice of paint, wallpaper or furniture. Intrigued? Check out Ikea’s Place app.
The future of MR
Imagine being able to try out clothes before shopping for them online, testing out if a wallpaper will look good in your environment before applying it, or virtual labels around your home reminding you of where you left your keys.
Tech giants like Apple, Google, Intel, and Microsoft are already pouring resources on developing Mixed Reality technologies. Apple has patented many MR technologies, with Samsung and Facebook getting in on the ground floor. There’s a lot of buzz and excitement about Magic Leap’s MR application of projecting light straight into the retina.
The future of Mixed Reality may mean that we will only need a single device to replace our screens. This will bring forth new ways to create content, as well as new ways of consuming them. As with other great technological disruptions that have changed our current way of life, Mixed Reality will create more industries and more jobs.
Mixed Reality will unlock a future where our natural world and powerful digital information meets. In a few years, we will see it mature and be nearly invisible to the user, blending seamlessly into the human experience.
Interested in future-proofing your career by learning cutting-edge tech skills? Check out our online courses in Australia to find out more.
Recently, I attended my first Virtual Reality music concert.
Standing on my bed in my small city apartment, I found myself in a psychedelic, virtual environment, dancing and cheering a DJ on a virtual dancefloor. The performers were HEAVYGRINDER, and the application, TheWaveVR on the HTC Vive.
This was the third flagship show to debut on TheWaveVR — and it was seriously unreal. After removing the headset, I had the leftover feeling of being at a live music. I had formed a memory of attending a music concert with people.
This sense of presence and accessibility — being able tofeel like you’re really experiencing live music, without leaving your home — is reason enough to be optimistic about the future of music in VR.
In this article, I will explore additional reasons to why VR is set to impact the music industry in a massive way.
Before we dive in…
Here’s a quick differentiation between the two main types of VR music concerts:
360 VR Video: A real-world concert is filmed using a 360-degree camera. This type of VR concert has gained popularity on mobile-based headsets like the Samsung Gear VR and Google Daydream through applications like Next VR. These applications offer live VR concerts, featuring bands like Imagine Dragons and Third Eye Blind.
Fully-simulated VR: TheWaveVR is an example of a fully-simulated VR music concert. Performers embody avatars and perform solely within a virtual space (i.e. there is no real-world event). This type of experience is better suited to high-end VR headsets like the HTC Vive as interactive elements like dance are important.
I wrote this article with fully-simulated VR in mind, but it effectively applies to both. In the future, we may see overlaps between the two types, and they may ultimately blend into a single platform.
A new wave of social
Today’s music streaming services, like Spotify, focus on the listening habits of the individual to drive user engagement. Song selections, albums, and playlists are all curated towards YOU based on your previous listening habits. This encourages you to treat music as a personal experience, something that entertains you in isolation.
This is where VR has the potential to offer something incredible to music tech: social experience.
You’d be forgiven for thinking that VR itself is an isolating technology. The truth is, VR has the potential to be an extremely social platform, allowing users to connect, interact, and share experiences with each other from anywhere in the world.
Voice, body language, and interaction help users to feel the presence of other people, making them feel like they’re together and that they’re occupying the same space, regardless of where they are in the real world.
Adding social to the VR concert experience means that it becomes something you do with people — something you do with your friends. When you’re with people you’re more likely to have a profound experience and develop richer memories. These will become memories that you and your friends will talk about for years as something that actually happened: “Remember that time we saw Daft Punk in Virtual Reality?”
On a grander level, there’s something about being part of a crowd at a concert that gives you a powerful sense of unity. Being one in a crowd of people vibing and dancing in sync, heightens the elation you feel when listening to live music. You feel like you’re a part of something larger than yourself, which is an awesome feeling. VR could be the tool that allows people from around the world to come together and unify, by experiencing the same music together.
It’s like Guitar Hero, but re-imagined for Virtual Reality.
This is how I often describe Audioshield, a VR rhythm game that has you blocking incoming orbs to the beat of a song.
Here’s Dylan Fitterer, the game’s creator:
“Brandon Laatsch (Hover Junkers, Duck Season) had an idea that music works so well in VR because it can synchronize three senses: sight, sound, and touch. I think he’s onto something there. VR’s main strength as a new medium is accurate body inputs, so any game that leans into that is on the right track,” says Dylan.
Accurate body inputs are at the crux of what makes this game fun. The orbs are flying towards the player to the beat of the song and instinctively the player defends themselves with their handheld shields. Through playing the game, the user has been encouraged to move their body to the beat of the music. They’ve been tricked into dancing. You could say that Audioshield gamifies dance.
Once the player is dancing (whether they’re aware of it or not), they’re almost guaranteed to have a good time. The movement of dance encourages the release of feel-good hormones, which makes the player feel elated.
It’s no wonder that rhythm-based experiences like Audioshield and SoundBoxing are some of my most regularly returned to VR games.
TheWaveVR also utilises the gamification of dance through the use of ‘challenges’ — simple but effective tasks that encourage users to dance and interact within the scene (like Jump, Hand Ups, Make Some Noise, and Move It).
During the ‘group hug’ challenge, the crowd huddles together into a giant mess of avatars, awkwardly holding hug poses until the progress bar fills up. Challenges like this bring crowds together and help channel a sense of unity.
Subscribe to the experience
So, let’s say Virtual Reality is going to be the future of music tech. It’s going to be a cultural revolution that will have music-lovers worldwide rejoicing together through shared music experiences.
That’s great, but how will it be monetised?
The most obvious form of monetisation, drawing parallels with the real thing, is ticket sales. Steven Hancock, COO of Melody VR said at Midem 2017:
“The live streaming works in the essence that the second a gig or a festival or a concert sells out, you can switch to a live VR ticket with a limitless capacity, so wherever you might only be able to hold 10,000 people…, 100,000, 200,000 people could buy into that live stream as it’s happening.”
Melody VR has taken a slightly different approach to VR concerts. While TheWaveVR is a completely simulated virtual environment, Melody VR is more concerned with capturing real-world events, using 360-degree filming technologies.
What Hancock is speaking about here is using VR as a complementary experience to the real thing. Missed out on buying a ticket to an upcoming concert? No worries, here’s a cheaper ticket to the Virtual Reality alternative that you can experience in real-time. Of course, it won’t be as exciting as the real thing, but hey, it’s something.
Ticket sales could become the primary source of monetisation when VR concerts start to gain popularity.
The Weeknd, Coldplay, and Katy Perry have an upcoming performance that’s only available as a fully simulated Virtual Reality experience. Would you buy a $5 ticket, even just to see what it’s like? Now imagine this on a worldwide scale. Everyone tuning into the same concert experience from across the globe, each paying $5. And the artist only has to perform once, and they don’t even need to travel!
Another likely form of monetisation could be monthly subscriptions: $10 a month for access to unlimited Virtual Reality nightclubs, concerts, and music festivals. It’s not hard to imagine the Spotify of the future integrating VR concerts into their existing subscription packages in an attempt to outdo competitors.
Whether ticket or subscription based, VR music will be profitable through the leveraging of its worldwide accessibility.
Shooting cats out of cannons
‘Chocolate’, a VR music experience from the mind of Tyler Hurd, could be described as an audio-visual sugar high, or probably more accurately as a bite-sized acid-trip.
This psychedelic, otherworldly experience has you jazzing out to music, shooting cats out of your canon hands, and watching them fall in slow-motion. It’s pure bliss.
Like Chocolate, some things are simply impossible to experience without the use of a VR headset. And of course, some things would be frowned upon by society if enacted in real-life (like shooting cats out of canons).
The potential for boundless and uninhibited experiences is one of strongest draw cards for Virtual Reality, allowing people to exist in ways far beyond their wildest imaginations. This is especially true for music experiences, as exemplified by Chocolate and TheWaveVR.
I reached out to Adam Arrigo, CEO of TheWaveVR, to get his perspective on the potential of music in VR. He wrote back the following:
“When we started working on the project about a year ago, the problem we said we were solving was ‘live music is expensive and inefficient.’ Like other companies, we saw the main value of VR as providing greater access to live music experiences. When we actually dug into what was engaging and unique about VR, though, we found that the real value was creating experiences that are only possible in — and designed specifically for — VR.”
“We stopped thinking of how to retrofit existing experiences and media types to VR and really asked ourselves what type of music content should exist in this medium. How do we amplify what’s good about concerts, give the audience and performer superpowers, create totally new artistic experiences that could transport fans inside artists’ minds, using 3D graphics and interaction design? How do we make it feel like a dream and not a recreation of something that will always be better in real life?”
Arrigo goes on to write about how the affordances of VR open up limitless possibilities for both the performer and the audience:
“The performer can be 10 times the size of the audience, be made of swirling particles that explode to the beat — they can transform the venue with the click of a button. And the audience? Instead of clapping, they can throw a giant pulsing star into the sky, or they can create their own light show that others can contribute to.”
Live music in VR should not aim to replicate the existing concert experience, but should instead aim to enhance it. Each performance could be something unique — something magical and foreign that users could not experience otherwise.”
Arrigo rounded off the email with this:
“We all just need to keep an open mind and keep pushing the envelope of what’s imaginable, because there’s so much incredible stuff coming in the years ahead.”
An actual real-life nightclub
We’ve talked about sociability, gamification, monetisation, and unimaginable experiences, but we’ve overlooked the most important facet of this investigation, and the most important consideration of any new product: What do the users think?
I reached out to TheWaveVR community to gauge user opinion. I received a bunch of comments, but this line from TheWaveVR user Lavender stood out:
“Every time I’ve been to a main headline event, I’ve come away with new friends and memories, and I get the same energy and groove as I do going to an actual IRL nightclub.”
This opinion, widely shared amongst the community, is the main reason I believe that Virtual Reality will be the future of music tech. If our relatively basic VR technology can replicate the feeling of going to a nightclub or concert now, can you imagine what the future of live music experiences could hold? It will be something truly incredible!
Get all the latest Academy Xi industry news, course updates, offers and more
Academy Xi acknowledges Traditional Owners of Country throughout Australia and recognises the continuing connection to lands, waters and communities. We pay our respect to Aboriginal and Torres Strait Islander cultures; and to Elders past and present. Aboriginal and Torres Strait Islander peoples should be aware that this website may contain images or names of people who have since passed away.