How real can computer graphics get?

Footer area

"My sweetheart!" A quote from the peculiar creature Gollum, who fascinated the moviegoers of the Lord of the Rings trilogy. Can you disguise someone like that? In fact, Gollum was created on the computer. An actor in a special suit equipped with numerous sensors played the role. Dozens of cameras recorded the scenes in specially prepared and lighted studios. The actors then had to be converted into the character of Gollum, frame by frame, in the computer, which took months of work. The cost was so enormous that it was only possible in a Hollywood production of this size.

Perfect animation

Figures in computer games are correspondingly unrealistic, albeit increasingly better. As a rule, they are completely generated on the computer. Whether it will ever be possible to depict movements so realistically that you don't need a really moving person as a template is in the stars.

In this respect, the question arises for science whether the cost of Hollywood productions can be drastically reduced. Professor Christian Theobalt from the Max Plack Institute for Computer Science in Saarbr├╝cken has achieved a breakthrough here and received the EU research award endowed with 1.5 million euros for his "CapReal" project. The Federal Ministry of Economics awarded the project one of the main prizes in the "IKT innovativ" competition.

Math replaces effort

Unlike in Hollywood, he succeeds in animation using a mathematical process. Few commercially available cameras record people against any background. For this to work, the computer needs to know some things about the objects and the light in a scene. This knowledge, together with the specific recordings, then creates a mathematical model from which the animation is derived.

In the first step, the person is measured and a digital skeleton with 19 joints is developed from it. The movements of the subsequent recordings are then transferred to the digital skeleton in real time. So you can immediately see a stick figure on the monitor that moves like the person recorded - and this from any perspective.

Finally, the skin and clothing are digitally pulled over so that the figure becomes more and more similar to the one filmed. Theobalt is convinced that his method will soon lead to animations that can no longer be distinguished from reality.

Capture and simulate facial expressions

Another important aspect is the three-dimensional face simulation, previously only possible in Hollywood with enormous technical effort. Here, too, the Saarbr├╝cken researchers, together with experts from other institutions, have developed methods to transfer faces to individual muscle groups of the face into the computer using the simplest of technical means. This allows the computer to generate lifelike facial expressions. In this way, one person's face can be recorded and transferred onto another's face.

Certainly a great prospect for cinema fans, but does the development also have benefits for everyday life and other areas of application? In fact, there are already a lot of ideas. One of them is the virtual mirror. The customer looks at himself on a monitor, but does not see his reflection in the mirror, but a simulation with different makeup, new glasses or changed clothing.

Ball watches the tee shot

For sport there is the possibility to record movement sequences exactly three-dimensionally. Details of the process can then be viewed in detail on the simulated figure from any angle. "The teeing off of a ball in golf could be shown from any perspective," says Theobalt, "if necessary from the perspective of the ball as well."

Another method developed by Theobalt's working group relates only to the hand and an extremely detailed recording of the hand movement. In this way, the computer records the smallest movement sequences and interprets them. The three-dimensional recorded movements of the hand can control a machine or computer without having to touch a keyboard or a screen.

Detailed mathematical analyzes of a video enable various changes to be made to the film, not just for animations. It can be important for a film to eliminate unwanted interference. A novel process can be used to remove people from a film, including any shadows. The possibly moving background is perfectly reconstructed from the video sequences in which the disturbance does not yet exist or no longer exists. In a sample film you can see musicians on stage. Someone is walking through the picture in front of them. Nothing of this person can be seen on the graphically reconstructed video.

Simulations for industry

Graphic processes are also of great importance for industry, as Professor Philipp Slussalek from the German Research Center for Artificial Intelligence (DFKI) explains. The "Visual Computing" research area he heads is about accurately modeling and simulating the real world. From this, predictions can be made about planned changes and their effects. The research area "Agents and Simulated Reality" uses very large, highly detailed 3D models to model interesting scenarios. It can equally be about complex production lines, ancient cities or biological cells.

Slussalek shows with light and shadow that this is not that easy. Two competing mathematical methods come up with different representations, neither of which adequately reflect reality. It was only through a combination of both methods that the researchers achieved a breakthrough in the simulation of certain light and shadow situations.

Virtual prototype

How important such a thing can be can be explained when introducing a new car model. The development effort is so enormous and the construction of prototypes so expensive that the new model is simulated first or only on the computer. However, this involves very tiny details, such as the shadow cast by the edge of the door. If it is not possible to perfectly simulate these shadows and the view of the vehicle, undesirable developments worth millions can result.

An application example of the research work is the simulation and visualization of production processes. Due to the increasing individualization of products, set-up and turnaround times have to become shorter and shorter. This can be achieved through three-dimensional simulations, since changes are played through and practiced in advance without interfering with ongoing production.

Unimagined possibilities

It is obvious that the planning of new settlements is much more successful if you can walk through them virtually. This requires optical simulations that are as perfect as possible.

All of these are just examples that show the importance of the computer for different graphic applications. The prospects for the future can hardly be estimated. We can look forward to the fact that the cinema will transport us into ever more realistic dream worlds.