darren david, gui geek

Archive for June, 2007

Creating MultiThreaded UI with HostVisual

Dwayne Need has a fantastic post (with source code) on how to enable non-interactive Visual elements to run in separate threads. His summary:

In general, objects in WPF can only be accessed from the thread that created them. Sometimes this restriction is confused with the UI thread, but this is not true, and it is perfectly fine for objects to live on other threads. But it is not generally possible to create an object on one thread, and access it from another. In almost all cases this will result in an InvalidOperationException, stating that “The calling thread cannot access this object because a different thread owns it.”

[snip]

In this demo I’ve shown how to use the HostVisual and VisualTarget classes to compose pieces of UI from different threads. There are some limitations: namely that the UI owned by the worker threads do not receive input events. There were also some annoyances we had to work around along the way, but those proved to be fairly minimal.

Multithreading has been a big barrier for me, coming from the Flash world, especially since TMTOWTDI. BackgroundWorkers are very powerful and easy to get my head around, but seem to be task-oriented as opposed to long-running-process-oriented. Creating real Threads and understanding exactly what’s going on there is still a bit beyond my understanding, though. Can anyone recommend a reference that walks one through the basics of multithreading in .NET, assuming little to no knowledge to start with?

3 comments

More HP Interactive Canvas Videos

Here are some more videos of the HP Interactive Canvas (from Obscura Digital via YouTube):

Still waiting to get a hold of the video with Martha Stewart…

2 comments

HP Multi-touch “Interactive Canvas” Unveiled at D5 Conference

I’m finally free to talk about what I’ve been working on for the past month! I worked very closely with Obscura Digital in San Francisco to build what is, to our knowledge, the world’s largest multi-touch, multi-user touchscreen. It was installed for the D5 conference in Carlsbad this past week. The installation was commissioned by HP (via Goodby Silverstein Partners) and tied in to their “The Computer is Personal Again” campaign — a large “interactive canvas” that allowed anyone to “personalize” the display with their touch.

The final surface weighed in at 16 feet by 7.5 feet, and sat roughly 2.5 feet off the floor. I was responsible for implementing the UI, which was built in WPF. The app allowed users to browse through 5 years of D conference videos, photos, articles and quotes, in addition to a live music sequencer and real-time stock quotes. I’m still waiting for some high quality videos to come through, but in the meantime, you can get an idea of the interaction model and scale of the piece from these short phonecam vids:

Get the Flash Player to see this player.

Get the Flash Player to see this player.

The biggest challenge on the WPF side was developing a multi-touch input framework from scratch. All of the familiar “Mouse*” events were useless to me; I had to extract point data from a UDP stream coming across the network from the machine which was translating touch in to x,y coordinates. I’m actually very pleased with the solution, it proved to be very performant and easy to work with. At the end of the day, I was surprised at how much could actually be accomplished in this model, and it only served to get me more excited about multi-touch surface computing. Perhaps this warrants some future posts on working with multi-touch, as it certainly seems to be the flavor of the month right now.

Highlights of the conference included showing Walt Mossberg how to interact with it (his response: “Very cool.” which, by the way, is my first Walt Mossberg review, and I’m ecstatic that it was positive :) ), in addition to Martha Stewart (yes, that Martha Stewart). Mossberg also mentioned the project while onstage with Bill Gates and Steve Jobs, in the same breath as the iPhone and the Microsoft Surface initiative, which are both real, shipping products (mention at 5:50). You can’t get better exposure than that.

Oh yeah, the entire project was completed, soup to nuts, in 4 weeks. That includes building hardware from scratch (which to me is just mindblowing), and only adds to my respect for the creative and technical geniuses at Obscura Digital. Expect some better videos and more posts on this topic soon, but for now, I’m happy to just catch my breath!

16 comments