We have instrumented a pair of dancing sneakers to each measure 16 different parameters, including continuous pressure at 2 points in the sole below left and right toes, continuous pressure at the front of the shoe for pointing, dynamic pressure below the heel, bidirectional bend of the sole, orentation about the "magnetic vertical" (a 3-axis solid-state compass), tilt in two axes (a low-G MEMs accelerometer), high-G's/shock in 3 axes (a piezoelectric accelerometer), angular rate about the vertical (a vibrating reed rate gyro), height above electric field transmitters in the stage, translational position (sonar from 4 separate locations), and battery status (continuous level and discrete warning). All sensors reside on the shoe itself, together with a PIC microcomputer to sample and serialize the data, an FM wireless transmitter to broadcast updates (at up to 20 kbits/sec, giving us a 50 Hz state update from both shoes) and a 9-volt battery that lasts a day or so. This system has been used in several interactive dance performances (see below), giving dancers intimate control over a dynamic musical stream that is mapped onto the shoe sensor signals via a complex rulebase. It won the 2000 Discover Award for Technical Innovation in the Entertainment category. For more details, click below:
Joe Paradiso (direction, technology, sensors, electronics, sounds)
Kai-Yuh Hsaio (musical mappings, performance software, embedded software)
Eric Hu (electronics)
Kaijen Hsiao (music software)
Ari Benbasat (electronics)
Ari Adler (mechanics)
Andy Wilson (java data plotter)
Yuying Chen (original dance performance and related advice)
A photo of an original-version actual shoe (we made two of them)
A video frame of Yuying dancing with the shoes in the Wearables Fashion Show
A photo of Yuying dancing with the shoes in the Wearables Fashion Show
A photo of the 1998 version of the shoes, preparing for performance
A photo of the 1998 shoe card, used by many dancers
A photo of the 1999 shoe card: final revision, the hardware is now totally mature
A photo of the pair of 1998 shoes with and without protective cover
A photo and another photo of a juggler (Takei Minoru) using the shoes in performance at the March 1999 Tokyo Toy Fair
A photo of Takei Minoru relaxing with the shoes after a performance at the March 1999 Tokyo Toy Fair
A photo of Joe Paradiso, Mark Dampolo, and Ari Benbasat at the 2000 Discover Awards Expo during our demos in the tent at EPCOT. Mark is a dancer from Disney (the Magic Kingdom) who demonstrated our shoes while we were there on June 23 and 24, 2000. Here are a few photos (1 - 2 - 3) that we took of Mark working with the shoes down there. Here's another of Mark demoing in the Expo tent and here are a few (1 - 2 - 3) of Mark doing a brief onstage demo at the actual Awards event.
A photo of our dancer Damiano at the demo we ran at ISEA2000 in Paris during December 2000.
An MP3 Clip Part-1 and Part-2 about the shoe system done at the 2000 Discover Awards Expo as broadcast on the radio show "Pulse of the Planet".
A MPEG video clip of Yuying testing a very simple musical mapping after the show (October 1997).
A MPEG video clip of Yuying appearing in the Wearables Fashion Show (October 1997).
A MPEG video clip of Mia Keinanen trying the 1998 shoes out with a very early test mapping (September 1998).
A MPEG video clip of Brian Clarkson doing a gymnastics routine at the Tokyo Wearables Fashion Show, (during NIKOGRAPH in Nov. 98) while wearing one of the shoes.
A MPEG video clip of our collaborator (Byron Suber, who then taught coreography at Cornell) demonstrating the latest musical mapping, with all sensors engaged (January 1999).
A Quicktime video clip showing Jason Goodstone, then a dance student at Arizona State University (ASU), doing a live demo with the mapping above during a presentation at the International Dance and Technology (IDAT) conference, on February 27, 1999. This is video shot from a camera in the back of the hall taking a microphone audio feed - while it's not the best vantage, it's a good performance that shows the system and mapping off nicely. Thanks to the Arts, Media, and Technology Program at ASU for digging this archival footage up for us (here in 2004!).
A Quicktime video clip from the same shoot as above, except this one shows Joe Paradiso giving a live demonstration of the various signals coming out of the shoe. Everything was working great, except perhaps the right pressure sensor, which was a little glitchy due to a failing FSR sensor, although it still produced useful data, as noted in the performance above.
A MPEG video clip of a juggler (Takei Minoru) using the shoes (w. the above mapping) for performance at a Media Lab exhibit during the March 1999 Tokyo Toy Fair.
A MPEG video clip of Boston dancer Dawn Kramer improvising with the above shoe mapping (April 1999).
A MPEG video clip showing portions of the performance that we did at the July 1999 American Dance Festival. The dancers were Rachel Boggia and Lena Rose Magee (each of which wore one instrumented shoe), except for a brief segment at the end showing Theresa Howard dancing with both shoes. Coreography was by Byron Suber of Cornell University, and portions of the piece featured music by David Borden, also of Cornell. The entire piece ran about 10 minutes; this is a clip of edited highlights. All sounds, samples, and sequences were triggered by the dancers.
A MPEG video clip of New York coreographer Mark Haim improvising with the shoe system at the July 1999 American Dance Festival.
A MPEG video clip of New York coreographer Mark Haim improvising with the shoe system at the Media Lab's Sens@bles conference in Kresge Auditorium, Oct. 20, 1999. This was a great performance; the shoe software was nicely perfected, and Mark was able to push the system to its edge.
A RealVideo clip showing Mark Haim's entire performance at Kresge, from which the above MPEG was excerpted. Large version with smooth motion (10 Meg) or Small version with choppier motion (2.3 Meg).
A Quicktime video clip of Disney Dancer Mark Dampolo (Magic Kingdom) demoing the system for the audience at the 2000 Discover Awards Ceremony at Disney World in Orlando (June 2000). The mix wasn't the best here (sounded better in the theater), so I compressed the audio quite a bit. Actually, it sounds like they only recorded one channel of the stereo pair onto the videotape, hence many sounds are missing or very soft.
A Quicktime video clip excerpted from a TV broadcast about the Discover Awards Expo.
A big Quicktime video clip showing some highlights of Mark Haim's 4 improvisational performances that were given at the Boston Museum of Science on March 4, 2001. As we are now developing new technologies and devices, this is probably the last performance that we'll be doing with this system and this musical mapping.
OK, due to public demand, we brought this rig out one more time, for a performance at the Ars Electronica Festival 2001 in Linz, Austria during September of 2001. This short Quicktime video clip shows Viennese dancer Chris Haring imrovising with the shoes at a rehersal prior to the show. The software was stretched to the breaking point with several unstable patches, but Chris still did a great show in the end (find it here on the Ars site).
A paper describing all details of this system, as published in the IBM System's Journal:
Design and Implementation of Expressive Footwear. J. Paradiso, K. Hsiao, A. Benbasat, Z. Teegarden, IBM Systems Journal, Volume 39, Nos. 3 & 4, October 2000, pp. 511-529.
The original paper describing our design concept at the First IEEE Wearables Conference:
Expressive Footwear for Computer-Augmented Dance Performance. J. Paradiso and E. Hu, in Proc. of the First International Symposium on Wearable Computers, Cambridge, MA., IEEE Computer Society Press, Oct. 13-14, 1997, pp. 165-166.
A paper discussing experiences with this system, presented at ISEA 2000, Paris:
FootNotes: Personal Reflections on the Development of Instrumented Dance Shoes and their Musical Applications. J. Paradiso, in Quinz, E., ed., Digital Performance, Anomalie, digital_arts Vol. 2,Anomos, Paris, 2002, pp. 34-49.
A paper describing the 1999 system, as used at the American Dance Festival:
Interactive Music for Instrumented Dancing Shoes. J. Paradiso, K. Hsiao and E. Hu, Proc. of the 1999 International Computer Music Conference, October 1999, pp. 453-456.
A paper describing the shoes and their performance:
The CyberShoe: A Wireless Multisensor Interface for a Dancer's Feet. Joseph Paradiso, Eric Hu, Kai-yuh Hsiao. Proc. of International Dance and Technology 99, Tempe AZ, Feb. 26-28, 1999, FullHouse Publishing, Lethbridge, Alberta, pp. 57-60, June 2000.
A paper describing the 1998 system:
Instrumented Footwear for Interactive Dance. Joseph Paradiso, Eric Hu, and Kai-Yuh Hsiao. Proc. of the XII Colloquium on Musical Informatics, Gorizia, Italy, September 24-26, 1998, pp. 89-92.
Eric Hu's MEng Thesis on the early shoe hardware:
Applications of Expressive Footwear, Eric Hu, M.Eng Thesis, MIT EECS Department and The MIT Media Laboratory, May 1999.
The Homepage for the Responsive Environments Group
