Conveying Language through Haptics: A Multi-sensory Approach

Votes: 12
Views: 2057

We rely on communication to interact with the world around us. Typically, we communicate through visual and auditory sensory inputs; however, due to cognitively demanding tasks or physiological impairments, these inputs may be preoccupied or unavailable. The MISSIVE (Multi-sensory Interface of Stretch, Squeeze, and Integrated Vibration Elements) is a novel, wearable, haptic communication device that transmits language to users through the sense of touch. The device is worn on the upper arm and made up of three types of haptic actuators, each of which is capable of rendering either squeeze, stretch, or vibrotactile sensations. Using combinations of these sensations, we developed a large set of distinct, multi-sensory cues to serve as a “haptic alphabet.” We validated the use of the MISSIVE system for communication through a user study, in which each phoneme in the English language was assigned to a specific haptic cue. In less than 2 hours of training, users learned to identify 150 different words.

The novelty of the MISSIVE device lies in its ability to render a large set of distinct haptic cues in a small, unobtrusive area of the body. Using multiple types of tactile stimuli allows the actuators to be positioned in close proximity to each other with minimal perceptual interference. We have not found any other literature reporting the development of discrete, multi-sensory cues of this nature.

The device consists of two wearable bands:
The Distal Band is a low-cost Velcro strap that positions four vibrotactors on the top, right, bottom, and left sides of the user’s arm. 3D-printed cases secure the vibrotactors to the band and allow for adjustable positioning. Although high-end vibrotactors are used in the current prototype, they could easily be replaced by a number of low-cost alternatives.

The Proximal Band consists of a 3D-printed frame that houses two small servomotors that actuate the radial squeeze mechanism and the lateral skin stretch mechanism. One servomotor is connected to a non-elastic armband that tightens around the user’s arm to create the squeezing sensation. The second servomotor is connected to a 3D-printed, semicircular rocker that is pressed against the user’s skin and elicits a stretch sensation when rotated.

Currently, a low-cost power supply and a small control box are used to run the vibrotactors, and an Arduino microcontroller is used to drive the servomotors. In future prototypes, these peripheral devices can be replaced with batteries and integrated circuits to improve wearability.

The MISSIVE is the first known haptic device that employs multi-sensory cues to communicate language. There are many situations in which MISSIVE would enable users to receive messages when visual and auditory channels are unavailable. For example, MISSIVE could enable those with hearing impairments to interpret language or to assist an existing hearing device. It could also allow unimpaired individuals to receive text messages safely while driving. In this manner, the MISSIVE device could not only provide a safer method of communication, but also increase quality of life for many users.



Voting is closed!


  • Name:
    Alix Macklin
  • Type of entry:
    Team members:
    Alix Macklin
    Jennifer Sullivan
    Nathan Dunkelberger
  • Profession:
    Graduate student (Rice University)
  • Alix is inspired by:
    As graduate students at Rice University, my team and I work to create and implement haptic technology into society in order to assist humans. We focus on the design, manufacture, and evaluation of mechatronic or robotic systems to rehabilitate, enhance and augment the human sensorimotor control system. Specifically, we utilize the sense of touch through tactile components in order to make the interaction between humans and technology or robotic systems more symbiotic. Through our designs we aim to make complex tasks, like surgery or controlling emergency vehicles remotely, safer and easier with haptic feedback. We also aim to provide opportunities for individuals with physiological impairments to better engage in their surroundings. Our haptic communication device, the MISSIVE, is not only an innovative form of communication, but it could also be used as assistive technology for hearing and/or visually impaired individuals, which is a major inspiration for us.
  • Patent status: