Overview

Timeline

December 2018, 3 weeks

Responsibilities

UX Design, Unity3D Developer

Context

teamLab Internship in Tokyo

Redesigning an interactive projection at the world’s largest digital art museum

As a teamLab interactive team intern, I was tasked with redesigning their Aerial Climbing exhibit’s projection. I worked as both the interaction designer and Unity3D developer of this project. I programmed the prototype from the ground up, with the sole exception of the flower assets.

Final Demo

Prototyping a dynamic river on the paths people walk

In this simulation, the blue cylinders represent people walking through the exhibit. I designed and developed this prototype in Unity3D.

Context

teamLab Borderless is a world of artworks without boundaries

teamLab is an interdisciplinary art collective that explores the relationship between nature and humans through digital technology. At their Borderless Digital Art Museum, artworks can flow from one room in a fully immersive without boundaries.

Problem Analysis

The live, interactive projection looks like a pre-recorded animation.

The Aerial Climbing exhibit's original projection is of birds flying around the room and avoiding the visitors. The visitors have different colors projected onto them, so the birds will change into their respective colors when they fly nearby.

WHAT WASn’t WORKING

Visitors are looking at their feet and not the birds.

I first noticed that the visitors were looking down at their feet due to the instability of the swings. The birds are mostly on the walls, so the most interesting part of the projection is often missed. It therefore makes sense that the main visual feature of the projection should happen around the visitors' feet.


WHAT WAS WORKING

The connection to nature, illusion of depth, and clear tracking of the visitors all worked well.

The birds connect with teamLab's greater theme of nature. The darkness of the projection helps give the illusion that the swings are suspended higher in the air. The colorful circles marking the visitor's location shows that the exhibit is following their position, making the interactivity more apparent.

WHAT MAKES the exhibit UNIQUE

Finding unique traits amongst the many teamLab projections

Most teamLab exhibitions use computer vision to track visitors position in the gallery space, so it was important to focus on the unique qualities of this particular exhibit.

The exhibition has set routes that people can travel down, which is very different from the other exhibitions where people can walk around anywhere in the room. People also generally move forward in one direction, the projection could encourage people to move forward and flow through exhibition.

the idea

Leveraging exhibit paths to create a dynamic river

To emphasize the branching paths and align teamLab's greater interest in the interaction of humans and nature, I proposed creating a flowing river where people walked.

The flow of the river increases as more people walk along the given path, and decreases overtime if no one visits it. As they walk, flowers will bloom along the sides of the river, and then disperse as they pass.

Improving user engagement with:

Eye-Catching Motion

The most exciting part of the projection happens where the visitors are looking.

Effecting Change

The visitors can see how they affect the projection with the changing flow of the river.

Continuous Feedback

The blooming of the flowers shows that the visitors' location is tracked in real-time.

IMPLEMENTATION

Simulating the Exhibit

I first had to make a simulation of people moving through the exhibit paths, since it wouldn't be practical to test with real people as I made it. The blue capsules represent the moving people.

Free-forming River

Though the simulated people walk robotically, the prototype is programmed so that the river dynamically forms no matter where the person walks. I've also add some noise to the river flow so it's more wavy.

"Loading” the River

The exhibit should have a river flowing through the paths before anyone starts walking on it, so I made it possible to save the river data and then load it when you start up the exhibition. You can also see here the river flow is stronger on the paths that the visitor walks on.

Adding Color

Each visitor is given a color. As they walk through the exhibit, their color will flow forward in the river. This makes it more obvious to visitors how they individually affect the exhibit.

Final Demo

Simulating a dynamic river with blooming flowers

The video includes three views: one long distance perspective, one from the visitor's perspective, and one from the top to see what the projection would look like. The flower assets were created previously by teamLab.


Reflection

Developing Emerging Technologies with UX Principles

This was a really fun project where I was able to tackle an experimental product. I learned how I could think about users interacting more subtly with the environment around them, even with limited ability to actually go out and interview people. I additionally learned about how there's many different ways to technically solve a problem— it was important to understand my high level role in the bigger picture and prioritize communicating my idea in the short 3 weeks.


Special thanks to my mentors, Jasper Degens and Sakiko Mikado, as well as all the others at teamLab!