AR Facade Moves Beyond the Lab

Renata writes "Researchers from Georgia Tech's GVU Center have installed AR Façade at the Grand Auto Text exhibition at the Beall Center for Art and Technology in Irvine, CA. The AR Façade installation presents an augmented reality version of desktop-based game Façade. The exhibit marks the first time this elaborate augmented reality interactive drama has been seen outside the GVU lab. The AR Façade immersive drama presents the virtual characters of Façade inside a real, physical apartment. Players play the role of an old friend invited over for drinks at a make-or-break moment in the collapsing marriage of the reactive characters, Grace and Trip. While some players attempt to pacify the characters, others break the ice with comic relief, performing for friends who can observe the unfolding drama from outside the exhibit. The uneasy social situation becomes all too real as players are able to move freely throughout a physical apartment and use gestures and speech to interact with the autonomous characters who appear graphically imposed in the space using a video-mix head-mounted AR display. The three month long exhibition will be open to public until December 15th."

Re:Technologically it's not quite impressive

[blairmacintyre]

While many of the comments are pretty amusing, I thought I'd reply to this one.

The registration between the characters and the world is negatively affected by a variety of things including
(a) the original facade rendering engine does not use real 3D perspective, and converting (due to the way trip and grace are procedurally rendered) would have been a huge undertaking; we spent some time on this and decided it wasn't going to be worth the effort, in the end. So they appear to float as they move away from the player and don't perfectly align with things like the door or bar
(b) the lighting it the room (at Georgia Tech) was not quite good enough for the tracker (which uses a camera to track markers on the ceiling); the lighting in the room at the Beall Center is much better. We didn't have the money to upgrade the lighting in the lab just for this experiment so it's a little jittery :)
(c) this is an open system, meaning the graphics, camera and tracker are not tightly synchronized (since the tracker is a separate commercial product running in parallel to the rest of the system). So there are slight misalignments there caused by the lack of sync

We are not doing any computer vision through the camera that is looking forward into the room; I don't do computer vision research, although it's obviously going to be important to integrate such things into a real system. Unfortunately, tracking natural features reliably (relative to a known baseline) and integrating those points with the rest of the system for stable pose estimation is not at the "cheap commodity" level yet, although some products are getting tantalizingly close (e.g., the Total Immersion stuff). If someone wants to solve that one and put it on SourceForge, send me a link. :)