Denny Unger On VR Mechanics, Multi-Modality, & the Inevitability of AR Glasses

This year marks the 10-year anniversary of Reality Labs, and we’re celebrating some of the developers who’ve made the last decade of magic possible. First, we interviewed Jesse Schell, a noted game designer, author, and CEO of Schell Games. Then, we caught up with Polyarc CEO & Co-Founder Tam Armstrong. Next, we sat down with Baobab Studios Co-Founder & CEO Maureen Fan. The following week, we shone a spotlight on Resolution Games Founder & CEO Tommy Palm. And today, we’re wrapping up our five-week series with a deep-dive conversation with Cloudhead Games CEO & CCO Denny Unger.

Cloudhead actually has a year on us, having celebrated its 11-year anniversary in August 2024. The devs behind early Rift epics The Gallery - Episode 1: Call of the Starseed and The Gallery - Episode 2: Heart of the Emberstone as well as action-rhythm fan-favorite Pistol Whip, Cloudhead is synonymous with VR innovation, having driven the industry forward with advances in locomotion and comfort. And we were stoked to borrow a bit of Unger’s time to discuss Cloudhead’s history, their groundbreaking work in VR mechanics, and what the future holds.

Denny Unger: The very first thing that drew me in was Virtuality’s Dactyl Nightmare in 1993. Playing that made me terribly uncomfortable because it was like five or 10 frames a second—I don’t know what it was, but it was bad.

Despite that, it’s really what spawned my initial excitement for the medium and something clicked for me. I could envision what the future looked like—our trajectory to VR and how it would continue to get better from there. I became kind of a basement VR builder, creating collimated displays, and obsessively hunting for the right mix of immersive gear. This led me to MTBS3D, a VR forum where I met Palmer Luckey, John Carmack, and a number of other VR luminaries who would then go on to work in the modern VR industry. Lots of conversations with those folks, and it was a really collaborative community where true believers would help each other with various VR projects.

Through that community, I ended up lending a small hand to Palmer with the Oculus Kickstarter approach, that would eventually become DK1, and even created the first Oculus logo. I saw an opportunity as soon as John Carmack got involved because he was so passionate about it. I thought, “This is going to blow up, and no one is paying attention.”

I decided to take funds from my prior business and essentially said, “Listen, we’re going to build the first adventure game for VR, and we’re going to commit 100%. We’re going to plant the user in the center of an interactive story. We are going to start development, even though we don’t have a complete picture. We don’t know when it’s going to launch commercially. We don’t know if there’s motion controllers. We don’t know anything. But we’re just going to start working.”

It was a bit of a leap of faith, but I was so passionate about how to shape new experiences in this medium that its future seemed secure in my eyes. I was taking the leap no matter what.

DU: Yeah, I think it was 2012, something like that.

DU: It’s a bit of a weird transition, but we built RPG gaming terrain across all genres, and dipped into board game design during that period as well. So the takeaways from that space were really about managing projects, building environmental assets and learning the nuances of creative collaboration. What was most valuable, I suppose, was the Wild West approach to game design and mechanics because board games are kind of all over the place. Anything goes, and you can be really creative with that, and I think that served our approach to VR really well.

Some of the people I worked with back at WorldWorks ended up following us to Cloudhead, so we’ve been working together for over 20 years!

DU: I had a brief period between WorldWorks and starting Cloudhead Games in a Flash-based advergaming studio. It was a bit soul sucking to be honest, but it taught me some important skills that made the move to video games easier. That was ultimately a big part of what prompted me to take the money I had earned from my last business and reinvest it in Cloudhead—and our jump into VR.

DU: Yes. Basically, every title we’ve created was really about discovering a specific aspect of VR design. For Call of the Starseed and Heart of the Emberstone, The Gallery Series, no one had cracked the hard problems around locomotion or motion-controlled interaction design. Narrative even in VR wasn’t really explored, and even performance bottlenecks weren’t explored.

So out of those titles came snap turns, teleportation, even how to understand your local play space—what are those constraints, and how are we going to visualize that in VR? No one had built anything like this yet, so it was essential framework for some of the first real VR games to market. We often take some of that earlier work for granted now, quite honestly. The first version of the dev kits that came out didn’t ship with controllers.

DU: And while early on we were developing for an Xbox controller, we thought, “No, hands are really going to be a thing.” So we used Razer Hydra motion controllers, which were fairly primitive at the time, to help establish how to give virtual objects weight, how to use your hands to interact with objects in VR, adopt poses, etc. A lot of innovation had to occur there. We sought to invent our way out of all these hard problems because no search engine was giving us magical answers and no one had yet paved the road.

We were also lucky enough to take the Portal franchise for a brief spin, developing Aperture Hand Lab to teach users how to manipulate their fingers before visual hand tracking was mainstream. Figuring out which fingers were actually important and how humans have adapted to using fingers was really interesting.

When the time came to work on Pistol Whip, we focused on creative solutions to get around the learning curve and friction points of VR. How do we rethink locomotion and input? Could we deeply simplify things to the point that a user gets what this game is in 30 seconds or less? We challenged ourselves to make a game that would be self-evident, driven by an instinct to survive and look and feel cool at the same time.

We’ve also dipped our toes into many other experiments and projects over the past 11 years, plenty of which have never seen the light of day. Because of the unique qualities of this medium, almost everything we explore is uncharted territory, which is always exciting.

DU: In the beginning, it was very stressful because once you put hand interaction or motion control interaction into an experience, you immediately recognize its value. So when it didn’t exist in those early days before CV1 territory, I kept asking myself, “Is this going to happen?” “The things that we’re working on, are they relevant?” “Will people care?” There was some self-doubt there, but ultimately, when you feel it in VR, you know it’s right. I’m very happy to be living in a future where hand input is an important part of VR design in general.

DU: I really look at MR as its own unique medium that perfectly compliments VR, but the rules are a bit different. In many ways, MR is finding its feet in the same way that VR has had to over the last decade. The hardware side often pushes new discoveries, which further drives the software side every six months, for many years.

The promise of MR for me is a bridging mechanism between completely simulated spaces and augmented reality spaces. Each of these mediums drives the value of the other in ways we still have to explore, but they’re both important and they both lead to a similar path in the road. But once these things become more mainstream and you’re wearing a pair of glasses, I think the value of being in a fully immersed reality vs. an augmented reality will become self-evident to people and they’ll want to do both—they’ll want to switch back and forth or find gradients in-between.

DU: Totally.

DU: In so many ways, VR, MR, and AR technologies require a full-stack of innovations and advancements to succeed. I personally find it difficult to pick a single innovation because it’s a bit like saying, “Pick your favorite child.”

Every advancement over the last 11 years has been important to the success of XR. Whether it’s 6DOF, motion control, standalone, pancake lenses, increased compute, or overall form factors, they all need to be there.

DU: I have so many thoughts on this. I believe one thing we’re learning in this space is that a one-size-fits-all approach does not fit all applications cleanly, because the use cases are as dense and complex as the world around us. That’s the thing about XR: It is truly a “super-medium” in that it can overlay onto almost anything we do in the real world. Whether for gaming, entertainment, fitness, education, enterprise, and beyond, there will be different idealized inputs and ways of interacting with that content. And the amount of options to engage with that content will likely increase, not shrink, in our future XR world.

Visual hand tracking makes a ton of sense for all kinds of use cases, but sometimes for accessibility you absolutely need haptics, buttons, sticks, and the reliability of tracked input, whether that’s for training, games, or other uses. You can’t just abandon certain input approaches without limiting what the technology is actually capable of.

We’re seeing an increasing kind of modality using XR hardware, so you might want to just sit on the couch and play a “flat” video game.

DU: You might want to do it on a flat screen that’s being projected from your PlayStation or Xbox. Keep the tracked controller and you're covering all bases, and you're not kneecapping all these studios that build for that approach to input modality.

DU: It’s a hard one for me to answer because in the beginning not too many developers or studios were paying attention to VR at all, so there was a really small cohort of developers sharing information and developing in a kind of void. It was a bit lonely, a bit scary, and you were trying to convince people of the value of this medium. In some ways we’re still doing that, but the public sentiment around the technology is far more mature—there’s a lot more attention and a lot more people in headsets who understand its value.

I’m also seeing a lot of re-exploration by developers who have discovered VR or MR for the first time, and they’re approaching it with a lot of excitement but with little guidance. I think there’s still a conversation to be had about best practices and how much we amplify these best practices for a new generation of VR, MR, or AR developers. I see a lot of wheel reinvention happening and/or I see a lot of best practices not being adhered to, leading to the creation of content that makes people uncomfortable or that’s difficult to use in general.

I feel like a re-education moment needs to happen soon because there is a groundswell of interest and the competitive landscape of XR in general. I feel like we have to blow the trumpet a little bit on best practices and standards.

DU: That’s the main reason everybody’s running on the hamster wheel, with little time and budget to explore solutions. Meta did set some really great best practice standards back in the day, and they probably did the most public-facing work out of any OEM in terms of trying to provide good guidance.

DU: This is really hard, because there’s been so many great moments. But during our early years of endless demos, this technology was so new and mostly unknown to many. We attended a conference to show off our latest version of Call of the Starseed around 2014 and we gave a few hundred people demos, but took for granted just how impactful the technology could be. One of the participants was in a wheelchair, and once the demo was over, they began to cry and said, “You have to realize, that’s the first time I’ve walked on a beach in decades.” It was in that moment that we knew how important VR could be for the world for reasons we hadn’t even considered. That one stands out to me.

DU: I have a strong sense that the tipping point for mass consumer adoption of VR/MR is just over the hill. We are increasing our velocity to key breakthroughs which will bring form factors down while increasing compute, creating an inescapable product that will replace not just your smartphone but your TV, your computer, your gaming console, your trinkets, and give you the ability to instantly connect with anyone across the globe as if they were with you in the same room. And we can also change that room!

I do feel like we’ve got a 10-year sort of spread, and I can see the throughline of how certain aspects of the technology are running faster and advancing faster than we thought they would, and some of them are just inescapable. You need that 10 years for things to bake, for software and hardware to mature, but at some point, no one is going to be holding a cell phone, and it will instead be a magical reality machine on your head all the time. I look forward to our collective super-powers in the near future.

DU: Our excitement for exploring VR as a medium hasn’t changed over the last eleven years. We’re hard at work on projects that, in some ways, expand into what more mainstream audiences might be attracted to in the space. There’s a bit of an unsaid, shared sentiment with OG VR studios to bring our collective experience and A-game to incoming titles.

We’re beyond excited to talk about it, but good things take time.

DU: I feel like XR has crawled out of a repeating negative press cycle and everyone working in this space should be incredibly proud for helping raise the tide. The people who’ve weathered the XR rollercoaster longest can clearly see the progress. In fact, investments in the technology at various OEM levels have only increased, and the competitive landscape has only gotten hotter.

It’s not a question of if it will succeed—it’s a question of when people will adopt it. Even if VR, MR, or AR hasn’t been an overnight success, its slow boil to mainstream is an inevitability, and eventually you’re going to be wearing glasses that do everything. There’s no stopping it.