48 Hours to Merge Neuroscience and VR

The GameTheory team was lucky enough to spend the last weekend of July doing what we love most: solving problems with games in entirely new ways.

We were asked to join Games for Change’s first ever Neuroscience VR Game Jam, or #BrainJam. And yes, that’s a quite a bit to fit into one title, but the event was outstanding, delivering on all those seemingly disconnected pieces to create one truly inspiring experience.

To understand this event, and what made it such a great weekend for the team, let’s break that title down. First it was a Game Jam. For those who don’t know, a Game Jam is similar to a hackathon, only where a product is the main goal of a hackathon, learning is much more important in a Game Jam. You take 24-48 hours and have to make a game from scratch around a specific theme. A bunch of people participate and you all tackle this same challenge together. At the end of the given time you see what you all managed to make.

In this case, neuroscience was our topic for the jam. While we like to think over here at GameTheory that we’re pretty wily, we certainly aren’t neuroscientists, so we had some provided for us in mint condition. We had Leanne Chukoskie from University of California San Diego and Patrick Beukema from the University of Pittsburgh who were amazing additions to the team.

The final catch to this Game Jam is that we were working in VR. All the teams were equipped with Vive and Oculus stations to create our games on virtual reality platforms.

The GameTheory team headed into Parson’s on Friday evening and got setup at our workspace at the New School. We were surrounded by inspiring VR development teams, neuroscientists, and researchers from all around the country. It’s always inspiring at Games for Change’s annual festival to see people from so many professions and walks of life coming together to use the power of games for positive change, and the Brain Jam was a fantastic example of that ethos.

Our team dove right in Friday night, working with our scientist partners to identify a variety of possible topics we could address with this platform. We thought about everything from PTSD treatment to motor skills training, but when we came back on Saturday morning to lock down our idea, we landed on top down processing as our inspiration.

We decided to make a game that was based around the idea of a Marco Polo type mechanic, looking for a hidden object using call and response cues. This is something our team scientists let us know is used a lot in research to see how people process information to find answers. The theme for our project bobbed right to the surface: a game where you play as a mother whale calling out for her lost baby and trying to find him.

And thus Marco was born! We spent the next two days rigorously creating in-game art, fish, seaweed, and ocean-scapes. We worked with our scientists to identify research scenarios and good ways to solidify the research testing environment the game would provide.

At the end of the weekend we, along with a dozen other teams who had come from around the world, presented our concept. We had made a game where you put on the headset and see out the eyes of this mother whale under the sea. You use your hand controls to paddle through the underwater scene, and use a button to call out to your baby. You baby answers but you have to listen carefully and move towards the sound. There are other distracting sounds like motorboats, dolphins, and seals, and lots of sights and creatures to avoid. This provides a much more interesting a realistic environment for researchers to use to assess top down processing. Researchers see how long it takes someone to find the baby and what tools they use to do that quickly and well.

We were thrilled that Marco was chosen along with two other games as the winner of the Brain Jam!

Something that struck us vividly at this event was how much benefit VR games can bring into neuroscience and research. Each team had identified a pile of tests that are currently done in research spaces that VR could prove to be a cheaper, better, more real-world tool for. By building games for these challenges we’re able to make research scenarios not only more interesting, but more affordable, and more relevant to science as they are able to recreate real-world environments far better than a simple set of lab equipment can.

The researchers we spoke to sounded so hungry for these type of innovations, and at GameTheory this is exactly the type of work we love to do: bringing academic experts and the insights of games together to create innovative solutions. It was thrilling for us to see such eagerness to adopt this new technology.

We’ve loved working with academics and researchers on our previous projects and creating with our scientist, gaming combo team at the Brain Jam just confirmed why we love what we do. We can’t wait for more opportunities to explore the future of research and virtual reality.