I’ve been building a half-sized version of Lumarca, an open source, low-cost, low-resolution volumetric display, and hooking it up to motion recognition systems as part of the research that I’m doing at SIAT. Lumarca’s creators, Matt Parker and Albert Hwang, have already hooked it up to a Kinect, which is great:
Here’s a couple of short videos of what my Lumarca system looks like.
Lumarca is a wonderful tool with so many subtleties. I’m realizing a few things about it as I’ve been replicating it.
Given a certain combination of distance between the strings and the projector and distance between the viewer and the strings, what may appear to be smooth movement by the graphics on my monitor is revealed to be quantized when projected onto the Lumarca strings. That is, pixels become more obvious. This isn’t a big revelation; we see this all the time when we project onto traditional, larger screens. One solution to adapt anti-aliasing techniques from 2D graphics to motion graphics projected onto Lumarca.
I’ve also been thinking about how visually striking the string I have been using is, and how it really can’t be ignored. A viewer can’t pretend that the strings are “not there”. We need to consider the materiality of the display, and doing so would open up new possibilities. For example, one could really play with the architectural properties of the strings. I think it should be possible in theory to build a variation of Lumarca that uses strings that cross each other at different angles or clumped in different ways.
Also, although 3D objects can be represented using Lumarca (as evidenced by Matt and Albert’s Kinect video above), I don’t exactly see 3D objects in the display, especially if a viewer stays in one spot with respect to the display. In the real world, we perceive 3D through a combination of strategies, including motion parallax and shadow+light perception (which is difficult in Lumarca because the audience sees only light and no shadows). There are a few potential solutions to this problem of creating a more 3D look in Lumarca graphics if applied for live dance/theatre performance:
Set up the performance so that the audience can shift perspectives. If you look at the Lumarca videos that Matt and Albert have online, you’ll notice that whenever the camera is moving around the display, that’s when Lumarca’s 3D-ness becomes most apparent. If that isn’t possible, have the display (or the audience) shift positions (either once in a while or constantly) to generate motion parallax. This isn’t always possible, of course, but it would certainly make a big difference if implemented.
Position the audience so that they are looking at Lumarca somewhat from above or below, not parallel to the projected light.
Create motion graphics that somehow give the illusion of shadows. This might be tricky given the nature of the display, but it’s an interesting challenge.
Simulate depth of field by simulating “blur” in the strings furthest away from the audience. I’m not sure how that would look like exactly, but it might be that the further back the strings are from the audience, the less saturated or less bright they are. Not sure if that would work, and you probably don’t need to do this a lot. Also, this approach means that the audience can’t shift positions, else the illusion would fail.
I’ve also been beginning to think of how one would build an immersive version of Lumarca where one can actually step inside it and play. Holes and pulleys and climbing rope figure in my designs.
Back to tying strings to nuts!
Update: I should point out that Lumarca is largely based on Albert Hwang’s project, Wiremap. It turns out that what I want to build is actually closer to Wiremap than to Lumarca.
Albert got in touch with me recently, and he’s thinking about visiting Vancouver and SIAT. Sweet. It turns out that 3D technologies for the stage are among his interests.
For my class in Computational Poetics, I’m looking into how somatic-based and phenomenological knowledge of human movement (I studied dance and computing science in my undergrad) can be used to inform the creation of new media-based live performance. As part of this research, I’m investigating the use of Lumarca—an open source, low-cost volumetric display created by Matt Parker and Albert Hwang at ITP—for use in theatre and dance performance. I was totally fascinated with what they had done, and I was thinking how cool it would be use Lumarca as the basis for a complex, custom-made theatrical set piece. Like equipping the strings with flex sensors so that physical interactions with the strings could be detected (for example, the strings could be regarded as vibratory bodies which could then be used for making and visualizing music). Or using different kinds of materials to create a very different look and feel, such as using beautifully-grained, polished wood (something we had discussed in my previous project, Biomodd [LBA2]). Or even eventually creating a theatre-sized version of the Lumarca/Wire Map so that dancers could be moving through and interacting with a sea of strings.
Basically, I’m tired of seeing 2D displays in dance performance, where they are projected on the floor, or on the ground, or rounded surface. I want to see/make something that surrounds the performer more!
What I especially love about Lumarca is how new media artists in resource-restricted contexts (such as in the third world; I normally live and teach at a university in the Philippines) can use it to create interesting work.
Construction on the Lumarca-based volumetric display is going well, though slow. I showed a super early version of it in a previous post, but I’ve come a long way from that. I’ve become recently become obsessed with making it as stable as possible. Not a bad obsession, and I justify it by thinking about how much I can save when I actually get around to positioning the strings. At any rate, this photo shows where I’m at with the construction.
I also have to write a paper around this project. So first I need to identify as many performances as I can that come close to what I want to do. For example, Improvisational Tools and Synchronous Objects come somewhat close, except that the viewer doesn’t actually get to experience these things live, in 3D space. I’ll look through the dance-tech maillist, my Delicious bookmarks, and YouTube for this.
Yesterday, I forgot/ignored the fact that the weekly TechSocial meet organized by Chris Demwell and his peers was happening the fabulous at Okonomi Sushi near Columbia station . I had a good excuse (or so I think): I had to look at an apartment at Fraser @ 43rd because my roommate and I have to move out of our current place by April. But without thinking, I suggested to my roommate that we go to Okonomi for dinner, where of course I bumped into the TechSocial folks. If I was trying to keep a low profile, I wasn’t doing a very good job of doing it. I fumbled awkwardly for a bit before Chris gave me a graceful way out by obliging me to update the group : I hadn’t accomplished what I said I would do, but I did submit a paper for the AAAI conference. Also, I started building my Lumarca system. Well, I bought the materials for building it. And I chose ultra-cheap, flimsy materials to work with because I wanted to do a quick and dirty prototype first and get a feel for the system. Chris was encouraging. “It’s a step in the right direction,” he said.
Tonight, I took a few more steps. I put together the frame. Tadaaa!
OK, it looks REALLY flimsy. But I just wanted to get the ball rolling. And I’m only building a super quick and dirty prototype first. There’s more to come, and I have big (Biomodd-inspired) plans for what to do with it when I build a new and better version of the frame.