Passive Acoustic Tap Tracking Across Large Interactive Surfaces
We have designed, built, and fielded a system that locates the position of knocks and taps atop a large sheet of glass. Our current setup uses four contact piezoelectric pickups (donated by Panasonic) located near the sheet's corners to record the structural-acoustic wavefront coming from the knocks. A digital signal processor extracts relevant characteristics from these signals, such as amplitudes, frequency components, and differential timings, which are used to estimate the location of the hit and provide other parameters, including the rough accuracy of this estimate, the nature of each hit (e.g., knuckle knock, metal tap, or fist bang), and the strike intensity. As this system requires only simple hardware, it needs no special adaptation of the glass pane, and allows all transducers to be mounted on the inner surface, hence it is quite easy to deploy as a retrofit to existing windows. This opens many applications, such as an interactive storefront, with dynamic content controlled by knocks on the display window. We have thusfar run this system on two types of glass - 1/4 inch tempered glass (which achieves a knock position resolution of about s=2 cm across 1.5 meters of glass) and 1/2 inch shatterproof glass (which achieves position resolutions ranging between s=3-4 cm across 2 meters of glass). Using an Analog Devices ADSP401 for realtime signal analysis results in a latency of only 60 ms per hit, so the response is quite rapid.
Although the position resolution is somewhat more coarse, the immediate benefits of this approach over touch sensitive interfaces include the large size of the active surface, low cost, intrinsic simplicity, optical transparency, high robustness, and the fact that there's no need to mount any hardware outside the window.
We first concieved this project was as a tactile interface for a virtual fish tank, but this was never realized. The first workable system in this family stemmed from a collaboration between Joe Paradiso and the the Tangible Media Group for the ball impact tracker with the Ping Pong Plus interactive ping pong table. Tracking the hits of hands on sheets of glass is much harder, however, because of the highly dispersive nature of bending waves that propagate through the glass and the poorly defined transients that can vary considerably knock-to-knock. As our techniques were progressively refined, the performance of the system improved. We have deployed five such interactive window systems in installations operating for the general public, one at the Ars Electronica Center in Linz Austria, one at the Kitchen Gallery in Manhattan, another at the American Greetings store near Rockefeller Plaza (also in Manhattan), another at the SIGGRAPH 2002 Conference in San Antonio, TX, and one at Motorola in Plantation, Florida. More detail on these installations are linked below:
Publications about this systemA paper describing this system in the journal Sensor Review
Tracking and Characterizing Knocks Atop Large Interactive Displays, Paradiso, J.A. and Leo, C.-K., in Sensor Review (special issue on vibration and impact sensing), Vol. 25, No. 2, 2005, pp. 134-143.
A paper presented on the system at the IEEE Sensors 2002 Conference
A short paper presented at CHI2002
A paper on interactive surfaces describing the early system in the IBM Systems Journal
A brief writeup for the Ars Electronica 2002 Proceedings
Description of the Interactive Window installation at SIGGRAPH 2002
Student Theses relevant to this project:
A System for Tracking and Characterizing Acoustic Impacts on Large Interactive Surfaces, Nisha Checka, M.Eng Thesis, MIT Department of EECS and MIT Media Lab, May 2001.
Return to the Responsive Environments Project Page
Che King Leo