Master’s Thesis Research

This is my Master’s research project at Carnegie Mellon University under the guidance of Jessica Hodgins. The project is set for completion by December 2004. See below for details…

Details | Screenshots | Movies

Technical Details
Project Overview
The goal of this project is to create a new technique for animating virtual characters using motion data and simulation in a context dependant way. The reason for such a hybrid system is that some animated tasks are better suited for traditional animation (stylistic behaviors) and others for simulation (interaction with the environment).

The aim is three-fold:

1. To allow character control to switch between physical simulation and kinematic motion data (keyframed/motion capture) whenever necessary.
2. Develop simulated behaviors that are more realistic than the limp and dead “rag dolls” seen in today’s videogames.
3. Reduce the reliance on motion data as researchers develop better techniques for simulating human behaviors. However, systems can still fall back on motion data for hard to simulate behaviors using this kind of hybrid framework.

The main challenges of this project are:

1. Infusing life into ragdoll simulation -
   Ragdoll simulation appears lifeless because it lacks control systems to generate human behavior. Research from robotics and biomechanics is applied to produce realistic simulated behaviors by modeling and controlling internal muscle forces. In particular, a robust fall controller is developed that can produce realistic responses to arbitrary forces applied to the body. New behaviors can easily be added as new simulation techniques are developed.
2. Quickly find closely matching frames in a motion database to poses generated by a simulation - The simulation changes the pose of the character considerably and cannot be predicted beforehand. A fast variant of the Approximate Nearest Neighbor Search (ANN) known as Majority-Weighted ANN is developed to quickly find similar motion data so that a transition back to the data can be made in real-time.
3. Ensuring that the simulation settles in a configuration that is close to existing motion data - This allows character control to smoothly switch to motion data. Proportional derivative controllers are used to allow the simulated human to use muscle forces to naturally push the body into configurations near those found using the fast ANN search strategy.

Here is a possible example of the high-level flow of activity in a videogame using this methodology:

UPDATE (11/13/04): The final version of the thesis document is available here.

UPDATE (11/24/04): I’m giving a talk on my thesis work at this year’s Game Developer’s Conference (GDC) in March. You can see the abstract here. I’m also the winner of the scholarship program for the Game Tech character seminar next week. I’ll be giving a short talk there on my thesis work. Also, look for my article on creating simulated behaviors using feedback control systems in the Game Programming Gems 5 book available in March.

UPDATE (2/26/05): I have posted materials (slides, demo, etc.) for my GDC 2005 talk on this work here.

Here is my current abstract for the thesis:

Versatile and Interactive Virtual Humans:
Hybrid use of Kinematic and Dynamic Motion Synthesis

Highly interactive characters that behave in situationally
appropriate ways are important for interactive entertainment
applications and virtual environments. For example, it would be
desirable if when a user is directing an avatar to sneak around an
obstacle, the avatar responds realistically to unexpected hazards
that may cause him to trip and fall. Realizing that no single motion
synthesis technique is perfect for every situation, we propose a
hybrid system that favors the one most suited to the current
objectives. Motion graphs are an effective tool for synthesizing
realistic and easily directable characters. However, because motion
graphs rely on splicing data obtained before runtime, they are not
adequate for applications where the external forces or detailed
interactions with the environment cannot be predicted. Simulation
allows the physical interactions between a character and its
environment to be modeled realistically, but does not provide a wide
range of behaviors because of the difficulty in constructing control
systems for complex behaviors. This thesis combines the unique
strengths of these two techniques so that complex animation tasks in
novel environments may be synthesized interactively. Our system
attempts to reasonably resolve when either technique is most
appropriate and provide the facilities to transition between them as
the character’s goals and interaction with the environment evolve.
To ensure these transitions are smooth, a fast variant of an
Approximate Nearest Neighbors search is developed to locate a good
correspondence between the simulation and motion database. Physical
controllers are used both to guide the simulation to the choice
points determined by the search and to add human-like responses when
simulating behaviors such as falling. We demonstrate the power of
this approach by synthesizing a variety of complex animation tasks
in real-time, switching between simulation or motion graphs in a
context dependant way. We evaluate our framework by showing how
motion graph controlled tasks may have simulated responses to
arbitrary forces, creating dramatic loss of balance, with full
recovery back to the original animated task. As simulation
techniques improve, such an architecture can support the future goal
of fully autonomous simulated characters, while still being able to
fall back on motion data for hard to simulate behaviors.

Early results after the first semester of work are described in more detail in this presentation.

More details of to come… (8/7/04)

Screenshots

Snapshots of the Fall Controller (added 8-7-04)

Comparison of Basic Ragdoll With Controller-based Fall:

Simulation (with controlled fall and roll) -> Motion Capture (added 8-7-04)

Updated Movies (physical controllers implemented)

A force is applied to a character causing a transition to simulation. A fall controller is activated that tries to put the hands in the way of the predicted shoulder landing position. The legs also kick up to cause a roll before transitioning back to motion data for the character to get back up. Download	Another fall, roll, and get back up example. Download
	More examples showing robustness of system to come….
Another fall, roll, and get back up example. Videos of other types of falls to come… Download

Preliminary Movies (no active controllers implemented)

Interactively selecting a joint and applying a force to it. Here, a force is applied to the head and a smooth transition to simulation is made. Fall is unrealistic until I implement physical controllers. Download	Another movie of interactively applying forces to a character. This movies shows a force applied to the upper leg joint. Fall is unrealistic until I implement physical controllers. Download

Example showing a seamless transition from running motion capture data to a simulation. Fall is unrealistic until I implement physical controllers. Download	Another example transition from running data to simulation. Fall is unrealistic until I implement physical controllers. Download

Transitioning from walking motion capture data to a simulated fall. Fall is unrealistic until I implement physical controllers. Download	An example of transition from a motion graph to a simulation. The approximate nearest neighber search is used to find the appropriate frame to allow the character to get up after the simulated fall. Fall is unrealistic until I implement physical controllers. Download

Another example of transition from a motion graph to simulation and back. Download