New Zealand’s first Mixed Reality Hackathon

New Zealand’s first Mixed Reality Hackathon

Quick Thoughts

4 April 2017 • Written by Maria Ayoung and Steph Beaumont

The ProjectR Mixed Reality Hackathon was the first of its kind in New Zealand – and a lucky few from our Wellington office had the opportunity to not only host the event in the Level 3 space, but also take part as Microsoft HoloLens testers and consultants. Amazingly, seven Microsoft HoloLenses were available for the participants to use throughout the weekend.

Starting at 4:30pm on Friday 17 March through to the evening of Sunday 19, Assurity hosted four teams comprised of a mixture of industry professionals and tertiary students, all interested in learning and creating a mixed reality experience, app or game for Microsoft HoloLens users. Unity helpers and people familiar with HoloLens were on site helping with the development of the teams’ concepts. Here’s some context to the event...

What’s a 'hackathon'?

Contrary to the connotations the term ‘hacking’ invokes, a ‘hackathon’ isa completely legal gathering where programmers (or, in this case, creatively minded people with little to no experience technically) are encouraged to collaboratively code and create in an intense way over a short period of time, typically over a 48-hour period. While working on a particular project, the idea is for each person or team to have the ability and freedom to work on whatever they want.

The hardware used

Microsoft HoloLens is the first self-contained, holographic computer. Transparent lenses and spatial sound enables the user to engage with digital content and interact with holograms in the real world around them.

Screen Shot 2017 04 04 at 10.17.27

The differences between AR, MR, and VR

Virtual Reality (VR) typically refers to computer technologies that use software to generate the realistic images, sounds and other sensations that replicate a real environment (or create an imaginary setting) and simulate a user’s physical presence in this environment.

Augmented Reality (AR) is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data. The recently popular Pokemon Go application is one great example of what can be achieved with augmented reality.

Mixed Reality (MR) is the merging of real and virtual worlds to produce new environments and visualisations where physical and digital objects co-exist and interact in real time.

The results

Screen Shot 2017 04 04 at 09.28.58

We were absolutely blown away by what each team achieved in such a short time.

Team aMaze created a very cool game. The aim was to find the meowing cat in the distance. The user was able to weave their way through a holographic maze, negotiating surprises hidden in the maze such as giant spiders and a zombie! This was an immersive experience with great visuals and sounds.

Team Wonder created a mixed reality, gift-giving treasure hunt experience whereby a gift giver would hide virtual items and keys within the physical space. The recipient then needed to find these items and keys to open a holographic treasure chest revealing a voucher to a real-world experience such as a high tea or a holiday to a Pacific island. Gift giving and receiving has never been more fun!

Team Ring created a wonderful visual tool for musicians and budding composers. Users were able to create drum beats from the visual rings that surrounded them. Going forward, this team envisioned an immersive experience with the user centred within a visual music editing and creation studio. This could make practising the piano a whole load more exciting.

Team Eden were the winning team of the hackathon. They created a fun and visually stunning mixed reality experience where a user is guided by a virtual bee to plant seeds and grow a garden by interacting with a rain cloud. Team Eden used sound effects such as rain falling from the cloud and an ambient track with birds tweeting to further enhance the user’s experience. Team Eden had quite a few great ideas for developing this experience further such as having a variety of plants and seeds available, animals attracted by the plants and a flame thrower to remove unwanted plants! Go to Team Eden’s blog to view their prototype and demo.

Playstation 4 Virtual Reality headset

ProjectR also set up a PS4 with virtual reality headset. Those lucky enough to try it out were not disappointed. As a user, you’re fully immersed and with the 360 degree audio, you’re quickly transported to a different world. The most popular game over the weekend was Ocean Decent where you are gently lowered down into the depths of the ocean in the ‘safety’ of a shark cage. In the beginning, it’s a truly magical experience while you marvel at the beautiful fish and coral scenery that pass you by. As you can imagine, this soon turns into a truly terrifying experience when you’re faced with an angry shark!

Screen Shot 2017 04 04 at 10.30.25

About ProjectR

ProjectR is a soon to be opened as a national virtual and mixed reality centre. This innovation hub is being set up in Wellington to provide a space for idea sharing, problem solving, development and testing of world-first uses for reality technologies.

So what’s next?

For us, we continue learning. There’s so much to know and discover in this new industry – so by being open to opportunities like this, we may well find ourselves testing for it. It’s time to get excited.

There are plans to run another ProjectR hackathon during Wellington techweek. So keep an eye out in this space and sign up quick because it will undoubtedly be popular.

The networks formed as a result of this experience will no doubt lend to more interesting Virtual and Mixed Reality endeavours in future. We’d like to thank Grant and the Assurity staff involved for letting us use the L3 space and for being a part of this awesome opportunity.


Observations from a weekend of working with Microsoft HoloLens

Prior to the Mixed Reality Hackathon weekend, we had little knowledge and zero experience in the practice of testing for any kind of Mixed Reality medium. Was it an exciting opportunity? Undoubtedly, yes. Did we anticipate how intense a full 48 hours of sharing a space with developers and creative people would be? We had no idea – and we (thankfully!) have no regrets. All that we observed, experienced and contributed to over this time has given us a much better insight into the emerging VR/AR/MR industry in New Zealand. Our key observations:

  • At a retail price of nearly $5,000, having the opportunity to handle the HoloLens headset felt like a surreal experience at first. The real fear of accidentally dropping and damaging something this valuable was very present in our minds. After a few practices though, this fear faded and placing the headset on felt natural and the adjusting of it second nature.

  • We struggled at first to establish our own direction when it came to consulting with the teams. We barely understood how the HoloLens worked. How could we test its usability and accessibility? How could we incorporate these into the early development stages of a product? What are we asking the teams to think about?

  • We found that many of the teams already had an idea of usability for their project, but few really knew what accessibility for their projects meant. So, while we stressed that thinking about who their audience was and what their audience would be anticipating/ looking for when using their product, the main considerations we suggested involved the limitations and possible accessibility issues their audience might encounter. These included colour blindness, hearing impairments and even spatial accessibility (in a 3D-holographic mixed reality setting, your real-world environment is a crucial consideration when developing a mixed reality product).

  • By the second day, we had defined an approach to working with HoloLens... and that was to use it, use it, use it. The more exposure we obtained through practice, the better we understood the potential limitations that HoloLens posed for its users. We concluded that, by being familiar with the hardware, we could put ourselves in the shoes of the target users and really empathise and imagine how the HoloLens product may be received.

  • Lastly, with the HoloLens, the users are interacting with 3D objects in their real-world environment. One of the most important usability observations we made about this was how applicable using a HoloLens in an office environment was. To put it very frankly, seeing a person wandering around or standing stationary while performing bizarre hand gesture, put the question in our minds; How usable is it? Would this headset fit in with a corporate professional office environment? Also, how is this going to change our work and social environments in the future? Is people behaving in this manner going to become the new norm – and should we be thinking of making the HoloLens more natural – aka something viable to corporate users who, at the end of the day, don’t want to look like ‘that one strange guy’ wandering around the office using tech the world is only just now accepting.

At the end of the day, our testing and consulting experience in the mixed reality scene is still in its infancy (a scene which a quick Google search can tell us has been around since 1994).

The main points we’ve taken from our experience is that usability and accessibility, even for this new industry, needs to be utilised from the early stages of developing a product through to the final iteration.

You just can’t underestimate the power of a solid, usable product. It’s what will keep the target audience coming back to you for more – and what will keep the fun factor of using the HoloLens past the initial ‘wow, this is so cool’ honeymoon stage. Ultimately, we now know much more about this space than what we knew when we went in. Bring on the future of mixed reality. We are *somewhat more* ready than before.


Search the Assurity website (Hit ESC to cancel)