NASA’s Langley Research Center has developed a technology at the forefront of a new generation of computer and video game environments that train valuable mental skills, beyond eye-hand coordination, for the personal improvement, not just the diversion, of the user.

The technology is derived from previous research focused on automatic adaptation of flight deck controls based on user feedback. Monitoring and enhancement of operator state is the objective of the current LaRC Intelligent Integrated Flight Deck Technology (IIFDT) program. New research has enabled modulation of the manual inputs that a player makes to joysticks, controllers, and image recognition video input devices. The new input device allows a player to control inputs to a video game by adjusting his/her physiological state, such as heart rate and breathing, in addition to standard controller or video inputs. This allows for an entirely new approach to integrating psychophysiological signals into game play.

The technology is constructed to allow modulation of a person’s input to a video game or computer simulation. Input modulation is based on the player’s psychophysiological state. The invention exploits current wireless motion-sensing and body gesture technologies to utilize physiological signals for input modulation. These include, but are not limited to, heart rate, muscle tension, and brain wave activity.

In addition, the experience of electronic game play can be enhanced by introducing
a multiplayer component in which various players collaboratively pursue the goals of the game. The device can also augment multiplayer experiences such as video game tournaments, in which the skill set required in competitive game play is increased by allowing players to interact with the game, and compete with one another, on a psychophysiological level. The current capability has been successfully prototyped using the Nintendo Wii console and wireless Wii remote as well as the Xbox™ console and Kinect™ motion-sensing device. Prototypes have been designed and are being developed to extend the current capability to the PlayStation™ Move™ and other similar game platforms.