HP Multi-touch “Interactive Canvas” Unveiled at D5 Conference
I’m finally free to talk about what I’ve been working on for the past month! I worked very closely with Obscura Digital in San Francisco to build what is, to our knowledge, the world’s largest multi-touch, multi-user touchscreen. It was installed for the D5 conference in Carlsbad this past week. The installation was commissioned by HP (via Goodby Silverstein Partners) and tied in to their “The Computer is Personal Again” campaign — a large “interactive canvas” that allowed anyone to “personalize” the display with their touch.
The final surface weighed in at 16 feet by 7.5 feet, and sat roughly 2.5 feet off the floor. I was responsible for implementing the UI, which was built in WPF. The app allowed users to browse through 5 years of D conference videos, photos, articles and quotes, in addition to a live music sequencer and real-time stock quotes. I’m still waiting for some high quality videos to come through, but in the meantime, you can get an idea of the interaction model and scale of the piece from these short phonecam vids:
Get the Flash Player to see this player.
Get the Flash Player to see this player.
The biggest challenge on the WPF side was developing a multi-touch input framework from scratch. All of the familiar “Mouse*” events were useless to me; I had to extract point data from a UDP stream coming across the network from the machine which was translating touch in to x,y coordinates. I’m actually very pleased with the solution, it proved to be very performant and easy to work with. At the end of the day, I was surprised at how much could actually be accomplished in this model, and it only served to get me more excited about multi-touch surface computing. Perhaps this warrants some future posts on working with multi-touch, as it certainly seems to be the flavor of the month right now.
Highlights of the conference included showing Walt Mossberg how to interact with it (his response: “Very cool.” which, by the way, is my first Walt Mossberg review, and I’m ecstatic that it was positive 
), in addition to Martha Stewart (yes, that Martha Stewart). Mossberg also mentioned the project while onstage with Bill Gates and Steve Jobs, in the same breath as the iPhone and the Microsoft Surface initiative, which are both real, shipping products (mention at 5:50). You can’t get better exposure than that.
Oh yeah, the entire project was completed, soup to nuts, in 4 weeks. That includes building hardware from scratch (which to me is just mindblowing), and only adds to my respect for the creative and technical geniuses at Obscura Digital. Expect some better videos and more posts on this topic soon, but for now, I’m happy to just catch my breath!
| Trackback16 Comments so far
Leave a reply
Hi, I am working on a similar project and will soon be working on a WPF framework for multitouch usingOSC over UDP.
I dont suppose that you can share any of your source code? or any tips that may make things easier?
Thanks and congratulations on a fantastic result
Hello, do you know what kind of display/touch technology was used? Are cameras used to detect touch points in a way similar to Jef Han’s method (http://cs.nyu.edu/~jhan/) ?
I get SWFObject undefined on this page
Hi i’m working on a multitouch graduation project right now in Sweden. This is interesting stuff. Nice job.
I have a blog with information on how to build a multitouch display as well, its located at http://www.multitouch.nl
sorry folks, videos should work now.
We used a modified version of the Frustrated Total Internal Refraction method to register touch. Our biggest problem was with blocking external infrared light, but we’ve got some tricks up our sleeve for v2.0 that should really knock touch sensitivity out of the park.
That’s awesome. I want to toss some videos around on the big screen.
[...] Wiz Darren David was responsible for putting together the user interface using Microsoft’s Windows Presentation [...]
[...] Wiz Darren David was responsible for putting together the user interface using Microsoft’s Windows Presentation [...]
Great work - we also earmarked Obscura as a potential partner to complete this project. We’re building on OpenCV and Touchlib. 16×8 seems to be the largest piece of acrylic out there - does anyone know where to get a larger piece? Also, does anyone know which projectors the used? I imagine it’s like obscura’s saturn piece?
[...] We used a modified version of the Frustrated Total Internal Refraction method to register touch. [...]
Wow, can you provide more details on what the modifications are? I guess you can’t use simple IR LEDs on this scale, you’d need hundreds.
[...] the fact that, according to Darren David, the interface designer of this particular installation, the software and hardware for the wall was built in four weeks, this could happen sooner rather than [...]
[...] Darren David alluded to this in his post but he's been so busy with the totally amazing HP Multi Touch kiosk he built (which you have to check out the videos for if you haven't yet) that he hasn't posted his [...]
[...] porting this to our WPF framework over the next few weeks and will be sure to share my results on the big screen. Thanks, Mike! Edit | [...]
Hi.
Good design, who make it?
[...] UI for General Motors for the Greenbuild Conference in Chicago. Similar in concept to the HP Interactive Canvas, Obscura engineered an all new rig that measured 18 feet x 5.5 feet, with 3 independent interaction [...]