.motion .bodies .communities .technologies

I’ve been building a half-sized version of Lumarca, an open source, low-cost, low-resolution volumetric display, and hooking it up to motion recognition systems as part of the research that I’m doing at SIAT. Lumarca’s creators, Matt Parker and Albert Hwang, have already hooked it up to a Kinect, which is great:

Here’s a couple of short videos of what my Lumarca system looks like.

Lumarca is a wonderful tool with so many subtleties. I’m realizing a few things about it as I’ve been replicating it.

Given a certain combination of distance between the strings and the projector and distance between the viewer and the strings, what may appear to be smooth movement by the graphics on my monitor is revealed to be quantized when projected onto the Lumarca strings. That is, pixels become more obvious. This isn’t a big revelation; we see this all the time when we project onto traditional, larger screens. One solution to adapt anti-aliasing techniques from 2D graphics to motion graphics projected onto Lumarca.

I’ve also been thinking about how visually striking the string I have been using is, and how it really can’t be ignored. A viewer can’t pretend that the strings are “not there”. We need to consider the materiality of the display, and doing so would open up new possibilities. For example, one could really play with the architectural properties of the strings. I think it should be possible in theory to build a variation of Lumarca that uses strings that cross each other at different angles or clumped in different ways.

Also, although 3D objects can be represented using Lumarca (as evidenced by Matt and Albert’s Kinect video above), I don’t exactly see 3D objects in the display, especially if a viewer stays in one spot with respect to the display. In the real world, we perceive 3D through a combination of strategies, including motion parallax and shadow+light perception (which is difficult in Lumarca because the audience sees only light and no shadows). There are a few potential solutions to this problem of creating a more 3D look in Lumarca graphics if applied for live dance/theatre performance:

• Set up the performance so that the audience can shift perspectives. If you look at the Lumarca videos that Matt and Albert have online, you’ll notice that whenever the camera is moving around the display, that’s when Lumarca’s 3D-ness becomes most apparent. If that isn’t possible, have the display (or the audience) shift positions (either once in a while or constantly) to generate motion parallax. This isn’t always possible, of course, but it would certainly make a big difference if implemented.
• Position the audience so that they are looking at Lumarca somewhat from above or below, not parallel to the projected light.
• Create motion graphics that somehow give the illusion of shadows. This might be tricky given the nature of the display, but it’s an interesting challenge.
• Simulate depth of field by simulating “blur” in the strings furthest away from the audience. I’m not sure how that would look like exactly, but it might be that the further back the strings are from the audience, the less saturated or less bright they are. Not sure if that would work, and you probably don’t need to do this a lot. Also, this approach means that the audience can’t shift positions, else the illusion would fail.

I’ve also been beginning to think of how one would build an immersive version of Lumarca where one can actually step inside it and play. Holes and pulleys and climbing rope figure in my designs.

Back to tying strings to nuts!

Update: I should point out that Lumarca is largely based on Albert Hwang’s project, Wiremap. It turns out that what I want to build is actually closer to Wiremap than to Lumarca.

Albert got in touch with me recently, and he’s thinking about visiting Vancouver and SIAT. Sweet. It turns out that 3D technologies for the stage are among his interests.

— Read more