Byte-Sized Tech Review — Learning from Video Games
Monument Valley, Ustwo Games
PLAY. WORK. BUILD. Interactive Projection Mapping, LAB at Rockwell Group
Ian Cheng, Interactive Installation, MoMA
The promise of experiential design is making spaces more interactive, immersive, and narratively rich. Video games seem to be an almost perfect virtual counterpart to experiential design, answering the same question: how do we design spaces and interaction in a way that tells a story and engages audiences?
In the first part of this series, I cover how video games make the most of their interactive systems in a way which most interactive installations do not. I was inspired to write this series based on the fact that for most interactive installations, you simply wave your arms in front of a depth sensor and particle systems swirl around you. This makes me sad. Interactive installations have some of the most advanced and expensive sensing and rendering systems out there, whose full capacities seem to be unused in the final design of these projects. Video games on the other hand, make do with cheap and simple input devices, yet somehow they use their limited input in a way which allows for experiences that are unique, interesting, and keep users coming back. The below examples cover how video games achieve varied gameplay out of limited input.
Monument Valley, Ustwo Games
Example 1: Monument Valley
Monument Valley is an interactive puzzler for iOS, inspired by the work of MC Escher. You move a little character around these hypnotizing isometric puzzles, twisting and turning objects to create impossible shapes and structures. The interactions are incredibly simple. Just tap on things or swipe on things. With only two inputs, this game still manages to both challenge the player and tell a compelling story while doing so. Monument Valley does this by putting the weight and complexity of the interactions on the game system, not the interaction system. It’s not the complicated inputs that make this game work so well, but rather the game progressing in a series of steps as the user interacts with it. From a technical perspective, this comes from the game programmers creating world states that are driven by the user: swipe 1 leads to game state A, which in turn leads to game states B and C, which snowballs into creating a web of discrete and unique worlds that the users themselves has effectively created. From a design perspective, the creators made a system that changes on a level that is not just aesthetic: there is a sense of progression with the interactions, a sense that the users decisions have weight and they are actually changing the game world in a meaningful way with their presence.
I bring up this example because it’s something I’d love to see in more interactive installations: evolving systems that are driven by the users in a way that is not just surface level aesthetic.
Journey, Thatgamecompany
Example 2: Journey
Journey was an adventure game made for PS4. Its premise was simple: jump your way from point A to point B. The entire interaction system can be explained with a single line: “the player can jump with one button, or emit a wordless shout or musical note with another; the length and volume of the shout depends on how the button is pressed, and the note stays in tune with the background music”. [1]
Why I bring up this example was the multiplayer approach for Journey. Randomly, a single other online user would be joined into people’s games and the only communication with this newly joined person would be the singing of your note. This joined individual could help the person jump higher and in doing so, complete levels faster and easier, adding motivation to continue communicating. That’s it. That’s the entire game. But when you ask people who played this game in its heyday about the experience, they get a wistful look in their eyes and smile back at you, and tell you there will never be anything like Journey ever again.
The interaction paradigm in this game was so incredibly simple, but once again, it does something I have not seen in interactive installations: designs that welcome multiple users whose individual interactions affect each other’s interaction with the world.
[1]: [https://en.wikipedia.org/wiki/Journey_(2012_video_game)]
Kerbal Space Program, Squad
Example 3:
Kerbal Space Program
This example is probably the least relevant for interactive installations, but covers the far, far end of what is possible with interactive systems. Kerbal Space program was a fully blown space program simulator. In it, a player learns how to construct a rocket ship and leave Earth. Every piece is simulated with consistent physics, and the result is complete user freedom in how they achieve their target task. The reason why I bring up this example is to illustrate what is possible within programmed environments. It’s possible to build an entire physics engine for your installation; simulating every piece and part can lead to emergent behaviors and users creating things you never could have expected.
Again, the interaction for this game is simple: point, click and drag. It’s amazing that within this limited interaction paradigm, a user can build an entire functional spaceship. This example is similar to the Monument Valley example with making a reactive game world. I am hoping for a world in which designers think about interaction for installations in terms of a reactive world, as opposed to a one-off interaction, which I really think could push physical installations with digital components to the next level.
Applying the lessons:
Redesigning PLAY. WORK. BUILD. by LAB at Rockwell Group
LAB designed PLAY. WORK. BUILD. at the National Building Museum in Washington, D.C., which was an interactive installation created to augment around Rockwell Group’s launch of a kid-friendly modular playground.
Play. Work. Build.
Kinect depth sensors drive interaction.
While the physical blocks themselves went on to be an extremely successful product, and sold almost a million copies around the world: the digital component of the exhibit fell by the wayside. While the design did incorporate some of the takeaways from this analysis, it didn’t commit fully to what it was.
The game system was a physically based 2D block system: which worked, but if we were to do this exhibit again, we would design something else. In the design process, instead of pulling examples from the real world, pulling designs from hit games from the Tower Builder genre could have helped us create a better engine and game design.
The Tower Builder genre games are deliberately designed not to rely on the often annoying parts of a 2D physics engine, but rather on user skill in navigating swaying towers that are more fragile the higher you get. A design like this would integrate the two main takeaways from this article: the first part is that users would have more meaningful impact on a game world, which would become more complex as the towers got higher and the space filled up. The second part leads from this, the multiplayer aspect suddenly becomes more important when you’re either competing with the person next to you, or collaborating to build something you couldn’t do on your own.
Summary/takeaways:
Build interactive systems that react to users on a level that is not just aesthetic. It’s possible to put complexity into the system, and use simple interactions to delight and drive meaningful change in the system.
Think multiplayer: how can you design interactions to include multiple people? Giving people the power to interact with one another through your interactive system tends to create entertaining and engaging results that you cannot get through programming alone.
By Brian Aronowitz, LAB Technologist