:: About :: Publications :: Hardware :: Software :: Related :: FlexiGesture
MIT Media Lab Responsive Environments Group

The software component of the FlexiGesture is responsible for allowing the user to explore the sound space, to train gesture-to-sound associations, to create mappings from input degrees of freedom to effect parameters, and to play with the finished result. During play, the software handles incoming data from the controller, performs gesture recognition to retrieve trained associations between gesture and sound, and manages the mappings from the continuous controls of the device onto the sound modification parameters.

  • C microcontroller code for gathering sensor data, some visual feedback generation, and serial data transmission
  • Java code for handling incoming sensor data, maintaining program state
  • Java/C code for gesture recognition