Multitouch: the state of the union.

A couple of days ago, Steve ended the dreadful 6 months wait after he announced that Apple was going to provide a full blown iPhone / iPod Touch SDK. Needless to say that I was among the people who caused the Apple Developer Connection servers to collapse by desperately trying to gain access to the sign-up page for the program. I was really impressed by the announcement and the general quality of the SDK that I got to experience.

Although, I was a bit sad to see that they still havent’ ironed out the issue of providing a clean object-oriented abstraction for multitouch gestures. That has been the source of my nightmares for the past year or so
Sometime before the iPhone came out last year, I was working on a small scale multitouch project involving the use of dot-matrix LEDs (see Jeff Han’s site to understand what I’m talking about). Unfortunately, I never got around to make it work since it required a deep understanding of microcontrollers and low level electronic circuits timings. I initially wanted to create a cheap reconfigurable multitouch USB device by lighting up specific LEDs designating “soft” sliders and switches. In the meantime, Yamaha came out with the Tenori-On, a musical synthetizer that also uses a grid of LEDs. I’m not sure that actually sensing touch using the technique demonstrated by Han and carefully described in this Carnegie Mellon paper.

Now, I ruminated many times over this problem of abstracting multitouch gestures, especially during the time I worked on my infrared LED-based mini multitouch table project last summer. I still wasn’t satisfied with my design until I saw the GestureMatch sample code provided by Apple with the SDK. Eureka ! The missing piece of the puzzle to complete the abstraction was basically there, in code form ! They are using cubic and quadratic Bezier curves to store the information and they use some geometry to check if the path closely fits the parameters of the curve. It is pretty rough and incomplete code, but seeing how this code works is enough to get ideas flowing in my head on how to generalize and add direction and timing support.