Cinematic Effects

Cinematic Effects

Gaffer to Game Engine: Cinematic Effects Kevin Bjorke Game Developers Conference, 2003 Cinematic Effects Technical What makes movie rendering different? Heavy-duty software rendering, <0.001fps Zero tolerance for obvious errors BUT, we can paint-out the bad frames in an emergency cant do that in a game! Massive scale Complex Models, Complex Surfaces

Complex Compositing may have hundreds of passes per frame Motion blur, shadows used everywhere Basis in Live-Action Traditions Techniques now approachable in real time Cinematic Effects Artistic What makes movie making different? Tightly-controlled camera Movie photography is as much about lighting as it is about camerawork Controlled timing animation, voice, music cues

Passive Audience Viewer identification with characters and events comes through indirect means not through experience in being or controlling the action Framing, staging, camera motion, lighting, and sound are all tuned to that very purpose drawing the viewer into the scene My Perspective California Institute of the Arts in cinematography Dr. Sid FFFP Dick Dastardley

Hanna-Barbera Wide variations in Realism: Toy Story, A Bugs Life, Final Fantasy, Flintstones, Jetsons, AniMatrix; live, cel, theme parks, TV, games Toy Story/A Bugs Life Disney/Pixar Time Machine of Dreams Sanrio Digital Productions, Abel, Omnibus, Kroyer Films, R/Greenberg NYC, dWi, AAA Paris, Lightmotive, Pixar, Square Home movies are real, but boring Realism (authenticity)

helps w/ suspension of disbelief, but just so we can get on with the main business Style and a unified look are ultimately more important Emphasis on illustration over simulation The Sixth Sense Disney, Spyglass Entertainment Style

Real Production Issues Its All About Control Directing the Eye Setting the Mood Saving Time & Money We Want a Big Toolbox! Big Toolbox, 2002 Design Chips Big Toolbox, 2001 Render Pictures Big Toolbox, 2003 CgFX puts the big toolbox

on your desktop. Real-World Lighting Big Tool #1 Live-action DPs/Photogs use lots of controls to manipulate the light and how it records Barndoors Scrims & Gobos Silks & Gels Reflectors Fog & Smoke Lab Effects Focusing Sources & Parabolics Lens Diffusion & Filtration

We want just as much control in games A Simple Example Simplicity Itself, Right? What Was Really Happening Over 50 lamps! And thats with real-time global transport, diffusion, and anisotropic BRDF algorithms already solved....

Tools for Any Kind of Lighting Job Which are most-important? The Most Important Lighting Tool of All Programmable Shaders Hold the Universe Together Simple Tools versus Compound Tools Gaffers Tape: Like duct tape, but with a key added feature: choice

Shape Light Shape is a volume, possibly with infinite bounds Pointlight = sphere Spotlight = cone What about a cube? How about an L-shaped prism? Shaping volume lets us direct the eye

The Limey Artisan Can be any volume we specify Color Variation Cg lets us choose any criterion: Variation according to surface orientation Complementary Lighting a la Shrek Diffuse GI Mapping

World-coords 3d location Light Maps Slide Projectors Negative Lights Variation helps us lend solidity to the scene Complementary Light Color float ldn = dot(Ln,Nn); ldn = max(0.0,ldn); Cl = lerp(Color1,Color2,ldn);

Shadows Shadows also define a volume. We can mix volume functions in Cg depth maps, polygonal stencil volumes, others Shadows dont have to be black lerp(shadowColor,lightColor,shadowfunc()); Shadows can be misdirected Light direction doesnt always have to be shadow direction the viewers eye will accept the shadow direction more than the light! Multiple sources can share the same shadow One source may have multiple shadows

Animatrix: Final Flight of the Osiris 2003 Warner Bros http://www.whatisthematrix.com/ Shadows Where They Dont Belong? Shadow as a Graphic Element Instantly Readable Jues contact shadow is posed for visual effect and to connect her firmly to the rooftop its not driven by the lightsources themselves. An Historical Example of Tweaking Carravaggios The Calling of

St. Matthew, 1599 Missing shadows? Where are the light sources, really? Fake Shadows with Gobos/Scrims Fake objects can be resolved mathematically Save on # of render passes Can stand-in for nonexistent models Can be texture-mapped

May consume shader parameter space, so be aware of the costs Scrims in Action Cast shadow based on scrim-coordinate z=0 Need to convert both Light position and direction to Scrims coordinate system Need to pass float4x4 Can be done by vp or fp Static positions can be precalculated by CPU Optional smoothstep()d edges X,Y at Z=0 can be used for tex2D() lookups

Scrims Can Have Arbitrary Volumes We can use cubes, spheres, or any other simple easy-toevaluate function, projected from the light or not Can incorporate 2D, 3D, or Cube textures Can apply to ambient or light maps too Varying Scrim Opacity Cg scrims can also be

inverted The Sixth Sense Disney , Spyglass Entertainment Useful feature for controlling attention: doorways, keyholes, etc Scrim + Shadow Map = Fader/Softener Mix shadow-map evaluation with deep scrim to create softshadowing effects Can apply to stencil shadows too We can selectively create

shadows, shape shadows, or selectively-suppress shadows Another Historical Example Caravaggios Martyrdom of St Matthew, 1599/1600 Hard shadows, or soft? Soldier: hard Saint: soft Hard shadows reflect Mrs. De

Winterss agitation, contrasted with the smooth calm light on Mrs. Danvers Rebecca 1940 United Artists. Courtesy MPTV Archives Creating Mood with Shadows Is it the light, or is it Bruce?

Good lighting isnt just technical showmanship the goal is to support the story The Sixth Sense Disney, Spyglass Entertainment Modern Example: SPOILER ALERT! Rim and Wrap Lighting Bypassing physics

makes lights more useful for games, allows us to use fewer lamps Contre-jour effects help to define shape FFP 2001 Special surface and/ or special light Wrap Lighting Simple method: angle L slightly toward the surface

tangent float4 wrapAxis = cross(-L,N); float rotAmt = acos(dot(N,-L))/(wrapAmount+(PI/2.0)); float4x4 theRotM = rotate_xform(rotAmt*wrapAmount,wrapAxis); // function L = mul(theRotM,L); /* and calculate diffuse and specular normally with this new L */ Even Simpler method: renormalize dot(L,N) float LdN = dot(L,N); /* normal lambertian would be = max(0,LdN) */ float adjVal = cos(wrapAngle); /* can be a precalculated constant */ float newLdN = (LdN+adjVal)/(1.0+adjVal); float diffuse = max(0,newLdN);

// alternative: float diffuse = smoothstep(-adjVal,1.0,LdN); Alternative Diffuse Lights: CgFX Demo Standard Wrap Combined Shadow-Inset Objects Use a slightly-smaller version of the geometry when making the shadow map/volume Can use the same geometry data and alter the vertex program to create a slightly-inset surface during the shadow-render pass float4 insetP = IN.Pos inset*IN.Normal;

OUT.HPosition = mul(WorldViewProjXf,insetP); The technique can be used to solve a variety of shadowing and shadow-aliasing (Z-fighting) problems Area Lights Extremely common, but very difficult to fully emulate Cg lets us select key visual attributes, mix &

match to get the idea across: Mix fake reflection w/soft diffuse Reflection Hack Softbox One-bounce raytrace to a scrim card Multiple cards can obscure one another Inverse-square law can be used

Can be used alongside cube maps FFP 2001 Textures/shadows can be added Cg Fragment Diff/Spec Example Manipulating Individual Parts of the Lighting Equation....

CgFX DEMO FFP 2001 Complexity Hidden in the Simple Composited glow helps define contour Scrim to emulate shadow of un-modeled window

Composited 2D artwork Softbox reflection of painted window to catch in eyes Diffuse-only bounces from non-existent walls Window is behind,

but light is from front okay because it reads in 2D Blinds shadow is painted texture Woo-method (midpoint) hard shadow from skin/shirt Blurry simple-method hair shadow from proxy object

Simple to implement once the methods & ideas are part of your usual work process Surface Shading Longer Pixel Shaders Give Us Lots of New BRDF Models Oren-Nayar diffuse Better Speculars Physically-based shading Etc. Etc. Etc. FFP 2001

Blue Felt Based on Gonioreflectometric Data Combining Appearance Models Complex ways to blend different kinds of surfaces via textures or procedures Use lerp() function freely on BRDF arguments and/ or on BRDF results Easier than ever to do in one pass No single BRDF, however flexible, will be right for all objects and shots, so Whatever works! Example: Skin

Textures are important, but so is the underlying shading method Only the face is textured in this image FFP 2001 Skin has many properties that tell us about the character and their life again, we want artistic control

Different Approaches to Skin Photographic / gonioreflectometric Hard to measure live people Hard to measure different areas Leaves little room for artist control Lafortune-Model Skin, Based on a real persons forehead

Different Approaches to Skin Analytical Attempt to model skin based on scientific theory Again, numerically-oriented and hard to control by artists Shrek Dreamworks/PDI Worked for Shrek though! FFP 2001

Kubelka-Munk layering, Pharr et al scattering, Jensen et al scattering Kubelka-Munk Scattering Different Approaches to Skin Fake It! Use simple model of skin layering and properties Depend on skilled artists for details Allow per-shot tweaking

FFP 2001 Make things approachable for the artists Whatever Works Fake Skin Multiple Layers Layers combined by varying versions of NV rolloff pow(dot(N,V),Q) = poor mans fresnel

FFP 2001 Varying Q among layers creates illusion of depth Special Surfaces Cg lets us shade arbitrarily for any purpose Custom Speculars Surfaces for compositing Non-photo-real (NPR) surfaces Baked lighting

Analytic Anti-aliasing Normalizsed ST Gradients: Color.xy = normalize(float2( ddx(IN.UV.s)+ddy(IN.UV.s), ddx(IN.UV.t)+ddy(IN.UV.t))); Raytracing in the surface When simple primitives are known, we can even raytrace in realtime Refracted rays strike plane perpendicular to object X axis Refraction and

intersection maths can be done by vertex shader Extra: Smoothstep() to threshhold specular term creates glossy surface effect Raytrace against textured virtual surface Cg Fragment Raytrace Example CgFX DEMO Compositing & Color Correction

Its all About Leading the Eye Contrast & color palette control, tone mapping Glares, Flares, Glows Mixing Elements from Multiple Artists Shadow Hacks Depth of Field Saving Time & Money Brother Where Art Thou Buena Vista Mood Control

Compositing & Color Correction Consider it for every frame! Amelie is a great example of end-to-end color manipulation ...and the DVD describes many of the details Amelie 2002 Canal+ Can alter color & feel without changing models

or the render pipeline Color Correction as a Texture Operation We can encode color corrections in 1D RGB textures, 256 elements long To color correct an image orig with correction texture cTex: half3 origColor = h3tex2D(orig,myST.xy); half3 newColor; newColor.r = h3tex1D(cTex,origColor.r).r; newColor.g = h3tex1D(cTex,origColor.g).g; newColor.b = h3tex1D(cTex,origColor.b).b; Unaltered colors

(channels expanded for viewing) Typical corrected colors (E6 film processed in C41) Color Correction Example Using the same correction texture Photo Kevin Bjorke Plain Original Cross-Processed Corrected

Useful References Architecture and Interior Design References Painters: Caravaggio, Rembrandt, Leighton, AlmaTadema, et al Cinematography: Reflections, Painting with Light, The Light on Her Face, Matters of Light and Depth, Lumieres et Ombres, Masters of Light, Film Lighting (Malkiewicz) Japanese Wonder Journal: Commercial Photo

Light Science & Magic, Hollywood Portraits COMMERCIAL PHOTO Magazine/Books TV, Films, Print Details far beyond any Euro/US publication Available at some western booksellers, e.g. Kinokuniya, Books Nippon

Also in Japan of course Useful References w/Exercises Light Science & Magic, Hunter & Fuqua Full of detailed descriptions of typical commercial practice for people and objects Interaction of Color, Albers Simply the best on the skills of evaluating and understanding color and

light perception. Originals with colored-paper inserts expensive and hard to find, paperback around 8 Buy some colored paper! Convergence Interactivity is changing movie production economics even as gaming rushes to supplant movies & TV Less Render Time == More Artistic Input Time Realtime will become important at the high-end, even if the technical gap never completely closes Artist time is more valuable than render time

Programmable shading lets you tape-together almost any desired effect Thats Show Biz! Thanks! [email protected]ia.com

Recently Viewed Presentations

  • Photo II - Currituck County Schools

    Photo II - Currituck County Schools

    Emotionalism. Expressionism. Formalism. 2. Criticism. Please list and describe the 4 steps of criticism: Research. Hypothesis. Describe. Calculate. Analyze. Opinion. Interpretation. Function. Judgement. 3. Elements of Art. List as many elements of art as you can: 4. Line. List the...
  • Social Identity and Public Health Practice - In class ...

    Social Identity and Public Health Practice - In class ...

    Social identity is an important aspect in power and politics. Frequently, there are social identity groups in a region or country that play a dominant role and others could be considered non-dominant. These dominance dynamics usually spill over into organisations.
  • How to Read the Bible

    How to Read the Bible

    It is important to commit portionsof the Bible to memory in Bible study. We should memorize God's Word, so we can . respond. according the Word when we are faced with temptations, troubles, and trials. Memorizing portions of Scripture will...
  • Using Music and Rhythm in the everyday teaching of English ...

    Using Music and Rhythm in the everyday teaching of English ...

    of English Language. in Malaysian Schools. Using Music and Rhythm. So, where are we going today? ... Using Chants and Rhymes. Nursery Rhymes. Ready-made chants. Making your own chants: 3. Using Simple Songs ... Using Music and Rhythm in the...
  • Origen e Historia de la Química

    Origen e Historia de la Química

    Origen e Historia de la Química Richard Robert Ernst 1991 (Switzerland, *1933-08-14) Desarrollo de alta resolución de espectroscopia de resonancia nuclear magnética (NMR), método de gran importancia para el análisis de estructuras moleculares Paul Crutzen (holandés), Mario Molina (mexicano), Frank...
  • Teaching Tips - Victoria

    Teaching Tips - Victoria

    Creativity(End: Privileging one "right way" rather than multiple pathways to one right answer) . Students should make, refine, and explore conjectures on the basis of evidence and use a variety of reasoning and proof techniques to confirm or disprove.
  • Body Mechanics - Humble Independent School District

    Body Mechanics - Humble Independent School District

    Body mechanics- using all body parts efficiently and in a careful way. These are necessary in order to limit stress and strain on the musculoskeletal system. Posture and body mechanics. Lifting, pushing, or pulling increases stress on the musculoskeletal system.
  • Learning to Plan: Planning Effective Literacy Lessons in ...

    Learning to Plan: Planning Effective Literacy Lessons in ...

    Learning to Plan: Planning Effective Literacy Lessons in English Language Arts Tell me and I'll forget, Show me and I'll remember, Involve me and I'll understand