Dynamic Visual Guides for Free-Gesture Musical Interaction


 

Free-gesture musical instruments, such as the Theremin, currently have no means of establishing position feedback except through the auditory channel. As a result, such instruments are notoriously difficult to play well, often requiring perfect pitch and a high degree of musical acumen. In this project, we have developed a set of motorized laser guides that serve as a set of visual aids in performing with such instruments. The lasers are mounted on robotic carriages (see below left), which can be dynamically moved along their track as the tuning of the piece changes. As well as providing a visual guide to assist in positioning one's hands, sensors in the carriages will detect the hand above the laser, quantizing the pitch if desired (while still allowing articulation about the center frequency). In addition, optical sensors looking along each laser beam can determine the continuous distance of the hand from the laser source, allowing other degrees of control when the hand intersects a laser. This instrument is an interesting hybrid of the Theremin, Laser Harp, and Dimension Beam, responding to both free and quasi-constrained gesture.

Link here for a Quicktime video clip (11.6 meg) showing a simple musical mapping of the Terminova in action written by Responsive Environments graduate student Nicholas Yu. The servos move to their reference positions at the start of the clip.

Link here for a Quicktime video clip (31.2 meg) showing a more complex mapping of the Terminova in action written by Responsive Environments graduate student Nicholas Yu. This mapping was demonstrated at the NIME 2002 Conference in Dublin

Link below for a paper overviewing the Terminova system:
The Termenova: A Hybrid Free-Gesture Interface Leila Hasan, Nicholas Yu, Joseph A. Paradiso. In Proceedings of the New Interfaces for Musical Expression 2002 (NIME-02) Conference, Dublin Ireland, May 24-26, 2002, pp. 122-127.

Link below for a Leila Hasan's M.Eng. Thesis detailing the Terminova system:
Leila Hasan -- Visual Frets for a Free-Gesture Musical Interface (pdf) June 2001.


Leila  Hasan



Joe Paradiso