Real Time Animation (RTA) is the interactive production of animated "cartoon" characters. It uses motion capture to track the movements and expressions of an actor, and uses these movements to animate a 3D cartoon character in real time.
When this RTA work was done (1996-1999), 3D computer graphics hardware was just emerging and the RTA work was state of the art and quite novel. Things have changed dramatically in the last few years, and today character animation is a routine part of video games.
The motion capture, consisting of both 3D body movements as well as facial expressions, was done with Modern Cartoons using a combination of Acension Flock of Birds and proprietary hardware and software for facial expressions. This work is still state of the art.
The linked video showcases an actress singing and dancing, animating the cartoon character "Alpha".
Another project combined RTA, a live actor and changing backgrounds, all combined in real time. This allowed the live actor and the virtual actor to directly interact with each other.
This project was a tremendous amount of fun, and stressed the capabilities of the hardware and software.
(click on either of the pictures above to run the video.)
How it was done: Motion Capture
Body Movements
Body movements were tracked using a combination of an Acension Flock of Birds and proprietary technology. The Flock of Birds uses a magnetic transmitter and a set of sensors. The system could determine the 3D location and orientation of each sensor. In order to provide realistic movements of the entire body, sensors were placed at the base of the spine, top of the spine, top of the head, and each wrist, elbow, ankle, and knee. Using a skeletal model of the character, this enable inverse kinematics calculations to determine the position and pose of the actor.
Animation of hands is a special challenge. Modern Cartoons developed proprietary technology which tracked the bending of each separate finger as well as the wrist, providing exceptional realism in the critical area of hand motion.
This picture shows an actress wearing the motion capture suit, with the animated character on the screen behind her. The highly reflective face tracking markers can clearly be seen.
Expressions
Providing a convincing character requires more than just body movements -- facial expressions are absolutely critical! Modern Cartoons developed a special optical tracking system which tracked expressions, including mouth movements (allowing realistic speaking), eye movements, and even eyebrows. This delivered an unprecedented level of realism and interaction.
Actress with optical tracking markers. The sunglasses are not worn during performance. Not shown are the optical and magnetic sensors, which can be seen in the picture above.