Tricorder: Mobile Interface to Sensor Networks

Joshua Lifton, Manas Mittal, Michael Lapinski and Joseph Paradiso

Research Sponsored by:
MIT Media Laboratory
Responsive Environments Group

The Tricorder
(A picture of the Tricorder GUI)
The Tricorder is a location, orientation, and network-aware handheld device used to interface in real time to a wireless sensor network embedded in surrounding domestic and occupational environment. As the name suggests, the inspiration for the Tricorder comes from the fictional device of the same name from the original Star Trek science fiction television series. The fictional version of the Tricorder was a self-contained device capable of sensing relevant information about whatever it was being pointed at (e.g., life signs 50 meters back, magnetic disturbance above, or plot thickener ahead). Our Tricorder device aims to achieve the same goals, but rather than being self-contained, our Tricorder pulls sensor data off a surrounding wireless sensor network.

The Tricorder Internals
(Tricorder Internals)

Physically, the Tricorder comprises a Nokia 770 internet tablet for display and user input purposes, a minimal radio node used to communicate with the surrounding embedded sensor network, a 3-axis compass to ascertain absolute orientation in three dimensions, a battery pack power supply, and a plastic case to hold it all together. See Figure 1. Like the fictional Tricorder, our version knows in which direction it is pointing thanks to the compass. This, combined with coarse localization based on the radio's received signal strength indication (RSSI) from nearby embedded sensor nodes, allows for real-time point-and-browse functionality while physically roaming within the sensor network itself.

The Tricorder was designed for use with the Plug sensor network, a collection of 35 power strips enhanced with sensing, wireless communication, and computational capabilities. The Plug sensor nodes are deployed just as regular power strips throughout the third floor of our lab. Each Plug sensor node can sense the electrical current drawn from each of its four electrical outlets, the electrical voltage supplied from the wall socket, sound, light, mechanical vibration, ambient temperature, and nearby motion. The data from these sensors are shared over a 2.4-GHz wireless radio network at up to half a megabit per second. The Tricorder's Plug-compatible radio can query nearby Plug sensor nodes directly or distant Plug sensor nodes by means of spreading a multi-hop request through the network and waiting for a response.

We've implemented a simple graphical user interface for the Tricorder. When using the Tricorder within the Plug sensor network, a map of the third floor is centered and oriented on the Tricorder's touch screen in real time according to the RSSI location estimate and onboard compass direction reading. Overlaid on the map are icons representing the Plug sensor nodes and their most recent sensor readings. The user can pan the map using the touch screen and zoom in or out using hardware buttons. Touching an icon reveals more detailed information about the corresponding Plug sensor node. The Plugs' multi-hop network allows for sensor data to be streamed even from far away nodes, in a sense granting the power to "see" through walls. In addition to communicating directly with the Plug sensor network, the Tricorder also has wireless Internet (IEEE 802.11) and Bluetooth capabilities, opening the possibility of accessing more traditional databases, websites, RSS feeds, etc. Figure 2 shows a screenshot of the Tricorder's graphical display. At present, the representation of sensor data is quite literal and direct; more abstract and interesting visualizations are certainly possible

The Tricorder GUI
(Tricorder GUI)

Further Links

A short paper on the Tricorder presented at CHI 2007
Lifton, J., Mittal, M., Lapinski, M., and Paradiso, J.A., "Tricorder: A mobile sensor network browser," in Proceedings of the ACM CHI 2007 conference, Mobile Spatial Interaction workshop, San Jose, CA, 28 April - 3 May 2007.

A video clip (79 Meg) showing our Tricorder in action. We begin with some brief inspiration from Star Trek, then show our Nokia 770 Tricorder in operation, then show the detailed response from one sensor node under various stimuli.

A short video clip (10 Meg) featuring our Tricorder excerpted from a TV show on Star Trek Technology. They got the basic idea right, but totally re-invented the Tricorder's UI in their own animation.

An article referencing our Tricorder project that appeared in the Boston Globe during July 2007

See also the content and papers posted under our PLUG and Dual Reality Lab websites.

copyright MIT Media Laboratory (all rights reserved)

Last modified: Tue Jan 30 15:17:52 EST 2007


Return to the Responsive Environments Group Projects Page