Franimation!

Last week I was biking around Crown Heights and saw a box of books on the side of the road. One turned out to be all about digital compositing, called “The Art and Science of Digital Compositing” by Ron Brinkman. It explains how digital compositing works from a technical viewpoint instead of an artistic standpoint and it’s been a really interesting read so far.

Here’s a quick description of how your computer combines a foreground and background image together. First off, you start with two images. A foreground image (in this case, a puppet) and a background image (in this case, the great pyramids).

puppet

pyramids

Images stored on a computer are broken up into three different images (called “channels”) that each represent a single color – usually red, green, and blue. This is often abbreviated as “RGB”.

Red, green and blue channels of the puppet image

Now, if you just plop the puppet image on top of the pyramid image, you’ll end up with a picture that only shows the puppet. Part of the puppet image needs to be transparent so the pyramids can show through. In order to do this, a matte channel needs to be created. A matte channel (sometimes called an alpha channel) is a greyscale image in which the opaque areas are shown as white and the transparent areas are shown as black. Matte images can be painted by hand or created inside the computer. In this case, the puppet was photographed against a red background so the background could easily be isolated and used to create a matte. As you can see in the color channel images above, the red background exists mainly on the red channel, while the puppet exists mainly on the blue and green channels.

puppet matte

The puppet image is now made up of four one-color channels of information; red color, green color, blue color, and a matte. Each pixel of each image ranges from dark to light. Usually these pixels are stored in the computer as a number ranging from 0-255, depending on the quality of the image. To simplify, some math is used to get the numbers to all range between 0-1. For instance, if the numbers start off ranging from 0-255, each pixel is multiplied by (1/255). Black pixels would end up 0, midtones would be .5, and so on.

Each pixel in the puppet’s color channels are then multiplied by it’s matte. In places where the matte is white, the image will be unchanged (original color x 1 = original color) while areas where the matte is black will make the puppet image black (original color x 0 = 0)

puppet

X

puppet matte

=

puppet multiplied by its matte

Next, the background needs to get a hole punched in it where we can put the puppet. The puppet’s matte is first inverted by a bit of math (1 – matte) so 0=1 and 1=0.

inverted mask

Then the background and inverted mask are combined in the same way the puppet was multiplied by its mask.

pyramids

X

inverted mask

=

pyramids multiplied by inverted puppet mask

Now we’re finally ready to put the puppet on the pyramid background. The numbers in the three color channels for the puppet are now added to the numbers in the color channels for the pyramid. Since the puppet ends up on the black area, puppet+0=puppet, and the puppet looks normal. Since the transparent area for the puppet is black, adding it to the background also has no effect.

puppet multiplied by its matte

+

pyramids multiplied by inverted puppet mask

=

puppet visits the pyramids!

There you have it. This is the process used to combine most foreground and background digital images, whether they be cars flying through explosions in Hollywood blockbusters or the arrow cursor on your desktop.