Description
Book SynopsisThe award-winning VES Handbook of Visual Effects remains the most complete guide to visual effects techniques and best practices available today. This new edition has been updated to include the latest, industry-standard techniques, technologies, and workflows for the ever-evolving fast paced world of visual effects. The Visual Effects Society (VES) tasked the original authors to update their areas of expertise, such as AR/VR Moviemaking, Color Management, Cameras, VFX Editorial, Stereoscopic and the Digital Intermediate, as well as provide detailed chapters on interactive games and full animation. Additionally, 56 contributors share their best methods, tips, tricks, and shortcuts developed through decades of trial and error and real-world, hands-on experience.
This third edition has been expanded to feature lessons on 2.5D/3D Compositing; 3D Scanning; Digital Cinematography; Editorial Workflow in Animated and Visual Effects Features; Gaming updates; General Geometry In
Table of Contents
Coming Soon …
Chapter 1: Introduction
First, Some Ground Rules
Now, the Introduction
Why Use Visual Effects
Chapter 2: Pre-Production /Preparation
Overview
Breaking Down a Script – Budgeting
Ballpark Budget
More Detailed Budgets
Bidding
Plate Photography
Temp Screenings
Reviewing Bids
Contracts
Rebidding during Shooting
Rebidding in Post
Monitoring the Budget and Schedule
Keeping the Budget Down
Working with the Director and Producer
Demo Reel
The Meeting
Moving Forward
Production Departments
Production Design
Camera
Working with the Cinematographer
Special Effects
Stunts
Wardrobe
Makeup
Production
Visual Effects
Editorial
Locations
Production Meeting
Designing Visual Effects Shots
Guidelines for Directors
Storyboards
Previs
Objective of the Shot
Concept Art
Continuity
Photorealism
Original Concepts
Budget
Reality and Magic
Camera Angles
Framing
Scale
Detail
Speed
Scaled Images
Depth of Field
Sequence of Shots
Camera Motion
Less Is More
Action Pacing
CG Characters
Creatures and Character Design
Powers of 10 Shots
Visual Effects Techniques
Technique Considerations
Additional Suggestions for Determining Techniques
Development of Previs Techniques
History and Background
What is Previs and Other Forms of Visualization?
What is Previs?
Different Types of Visualization and When to Use Them
Visualization: The New Essential
The Application of Previs: Who Benefits and How?
The Benefits of Previs
Project Types
Post-Visualization
What is Post-Visualization?
Why Use Postvis?
Who Does Postvis?
Cautions and Suggestions for Good Practice
The Perils of Previs!
Passing the Work On
The Role of the VFX Supervisor in Previs
Previs: Advanced Techniques
Visualization Usefulness
VR as a Tool
Visualization in Engine
Render in Engine
Visualization in Real Time
AR as a Tool
Camera Angle Projection
Introduction
Drawing What the Lens Sees
Techvis
What Is Techvis?
Who is Techvis for?
Virtual Production
What is Virtual Production?
How is Virtual Production Used?
Chapter 3: Acquisition / Shooting
Working on Set
Common Types of Special Effects
What are Special Effects?
A Brief History of Special Effects
The Special Effects Coordinator
Working with the Visual Effects
Visual Effects in Service to SFX
Special Effects Design and Planning
Storyboards and Previs
The Elements: Rain, Wind, and Snow and Ice
Smoke, Fire, and Pyrotechnics
Mechanical Effects
Flying Wire Rigs and Stunts
Safety
Greenscreen and Bluescreen Photograph
Best Practices and Otherwise
Overview
Function of the Backing
Negative Scanning and Digital Conversion
Backing Uniformity and Screen Correction
The Alpha Channel
The Processed Foreground
The Composite
Recommended Specifications and Practices
How to Expose a Green Screen Shot, and Why
Setting Screen Brightness
Choosing the Backing Color
Floor Shots, Virtual Sets
Foreground Lighting
Controlling Spill Light
Lighting Virtual Sets
Tracking Markers
On-Set Preview
Cameras for Blue Screen or Green Screen Photography
Underwater Photography
On-Set Data Acquisition
Camera Report
Tracking Markers
Props for the Actors
Cyberscanning
Digital Photos
Lidar/Laser Scanning
Lens Distortion Charts
HDRI and Chrome Balls
Lidar Scanning and Acquisition
On-Set 3D Scanning Systems
Types of Technology
Lidar
Photogrammetry
Prop Scanners
Lighting Data
Gathering Lighting Data
Beware of False Savings!
Using Conventional Still Cameras
Reference Shooting Considerations
Clean Plates
Shooting the Clean Plate
Locked-Off Camera
Moving Camera
Other Issues
Post-Process
Alternates without Clean Plates
Other Uses for Clean Plates
Monster Sticks
On-Set Animation Capture: Witness Cam (IBMC)
Wireless Non-Video Motion Capture
Factors Affecting Witness Cameras
Dealing with the Data in Post-Production
Camera Tracking for Real-Time Visualization
Camera Tracking Pre-Production
The Camera Department
Prior to Shooting
Current Tracking Systems in Use
Triangulation As a Method of Recording Camera Data
Camera/Subject Positional Information
Basics: The Toolkit
Basics: Nodal Point
Photographic Reference
How to Proceed
Shooting Video as a Reference
Rules, Setup, and Testing
Do a Complete Test Shot!
Why Run Through Example or Test Shots?
Digital Cinematography
Digital Definitions
High Dynamic Range (HDR)
Lens Metadata
Look Management
The Recording System
Lens Mapping for VFX
VFX Photography
The Camera Array
Designing an Array Shot
Technicians
Shoot Day
Special Techniques
Post
The Future
Filming Live-Action Plates to be Used in VFX
Camera Position (Station Point)
Angle of View
Lighting Considerations
Camera Tilt
Background Quality
Moving Plates
Scouting the Camera Positions
A Case Study
Camera Cars
Camera Car Safety Issues
Purpose-Built Crane Cars
Vibration and Camera Stabilization
Road Speed
Precautions
Panoramic Rigs
On the Water
Air to Air
Cable Systems
Shooting Elements for Compositing
What Is an Element?
Stock Footage
Types of Elements
Generic versus Shot-Specific Elements
Determining Element Needs
Cheating
Backgrounds
Black Backgrounds
Line-Up
Camera Format Considerations
Assorted Methods for Shooting Elements
High-Speed Photography and Filming Elements
Cameras
Technicians
Director of Photography
Lighting
Application
Locking Down the Camera
Video Assist
Post
Supervising Motion Control
What is Motion Control?
Performance Choreography
Multiple-Pass Photography
Scaling
Import and Export of Camera Move Data
The Data
Types of Motion Control Systems
Motion Control Software
Camera Types
Sync and Phase
Dealing with Production
Acquisition of Motion / Still Photographic Textures for Mapping onto CG
Panoramic Backgrounds
Tiled Stills
Motion Tiling and Synchronous Plates
Practical Considerations
Stills for Textures and Lighting
Stop-Motion
Evolution of Stop-Motion Photography
The Time Required to Shoot in Stop-Motion
Preparation before Shooting
Setting up a Shooting Space for Stop-Motion
Use of Motion Control in Stop-Motion
Useful Caveats
Evolution of a Shot
Use of Stop-Motion in Visual Effects
Chapter 4: Performance and Motion Capture
What is Motion Capture?
Is Motion Capture Right for a Project?
The Mocap Look
Technical Specifications
Entry Point
Budget
Which Technology is Right for a Project?
Gauging a Project’s Needs and Constraints
Passive Retroreflective Optical
Active Optical
Inertial
Structured Light
Dense Stereo Reconstruction
Bend Sensors
Preparing for Motion Capture
Actors
Motion Capture Suits
Marker Placement – Body
Marker Placement – Face
Rigging for Motion Capture
Shot List
Technology Considerations
Hardware
The Strobe
Markers
Lenses
Filter
Image Sensor
Onboard Processor
Inputs/Outputs
Setup
Software
Acquisition
Calibration
Post-Processing
Reconstruction
Labeling
Gap Filling
Cleaning
Solving Motion Capture
Facial Motion Capture
Facial Actor Survey
Actor Survey – Hardware
Reference Data
Statistical Data
Facial Rigging
Facial Acquisition
Audio
Facial Motion Capture Solving
Real-Time Motion Capture
Real-Time Uses
Real-Time Limitations
Alternate Technologies
Motion Capture Resources
Virtual Production
World Building
Previsualization
On-Set Visualization
Virtual Cinematography
Chapter 5: Stereoscopic 3D
How 3D Works
Accommodation and Convergence
Interaxial Separation
Toe-in Versus Horizontal Image Translation
Parallax or Depth Budget
Positive and Negative Parallax
Floating Windows
Fix It in Post
Stereoscopic Design
The Emerging Grammar of 3D
Creative Use of Depth
Previsualization
Avoiding Painful 3D
The Aesthetic of Scale
Cutting for 3D
Designing for Multiple Release Formats
Immersion-Based versus Convergence-Based Stereo
Native Stereo
Pre-Production
On-Set
Stereography in the Visual Effects Process
Stereography for Finishing
HFR as a Solution for Better 3D Movies
VFX Elements and Stereo
Introduction – How VFX Elements are Used
Native Stereo Content
Mono Capture – Packaged Script and Element Deliveries
Mono Capture – Hybrid Approach for Stereo Delivery
Mono Capture – Full CG Approach for Stereo Delivery
Creating Depth – Layout and Stereography
Stereo Camera – Depth Wedges
Stereo Compositing
Requesting a Full CG Stereo Render Mid-Production
VFX Production Tasks
2D to 3D Conversion
Depth Creation Preparation
Visual Analysis of 2D Depth Cues
Pre-Production and Previs for Conversion
Source and Target Perspective
Shared Shots/Shared Workflows
Main Stages of 2D-to-3D Conversion
Major 2D-to-3D Conversion Workflows
Special Cases
Re-projection Mapping Workflow
Pixel Displacement or Pixel Shifting
Other 2D-to-3D Conversion Workflows
Is "Real" Always Right?
2D-to-3D Conversion Management
Stereoscopic Visual Effects
Prepping for the Third Dimension
Shooting the Third Dimension
Visual Effects in the Third Dimension
Photographed Elements
Accuracy and Attention to Detail
Artistic Skill Level
Data Management
Stereoscopic Digital Intermediate Workflow
Stereoscopic 3D Process Milestone
Viewing 3D Dailies
Projection Screens for Stereoscopic Viewing
3D Editorial Processes
Data Workflow
Applying the 3D Grade
3D Stereo Deliverables
3D Home Video Deliverables
Stereoscopic Window
The Stereoscopic Window
Placement of the Window in Relation to the 3D Scene
Window Violations
Window Placement Logic
How to Create a Stereoscopic Window
Producing Movies in Three Dimensions
Development – Getting the Green Light
Production – What to Look Out For
Chapter 6: Post-Production / Image Manipulation
Resolution and Image Format Considerations
Formats
Transport
Resolution
Academy Color Encoding System (ACES)
ACES Components
ACES Benefits
ACES Color Space Encoding
Viewing ACES
Preparations for Using ACES
Image Compression/File Formats for Post Production
Image Encoding
Still Image Compression
Other Lossless Compression Methods
File Formats
Color Management
The Three Guidelines
Digital Color Image Encodings and Digital Cameras
Color Management at the Desktop
Bringing Color Management to Film Workflows
Digital Intermediate
Shot Element Pulls and Delivery to VFX
Introduction
The Lab
"Production" Is to "Lab" as …
The Merge
The Handoff
VFX Editorial
The Select
The Pull
VFX Editorial
Editing within a Shot: The Art of Precompositing (Precomp)
How It Came to Be
Modern Day Tracking and Disseminating of Information
As the Shot Changes
Wrapping It Up
Editorial Workflow in Feature Animation
Introduction
Editorial Crew Staffing and Structure
Editorial Involvement with Feature Animation Production Stages
Communication with Artists
Starting
Working with Teams
Working Globally
Reference and Perspective
Shot Production
Communicating with Artists in Other Departments
Completion
Compositing of Live-Action Elements
Modern Digital Compositing
Scene Tracking
Rotoscoping and Paint
Rotoscoping
Digital Painting and Plate Reconstruction
Matte Paintings/Creative Environments
Matte Paintings: Art of the Digital Realm
What Is a Matte Painting?
Matte Painting Pioneers and History
Visualizing the Matte Painting Shot in Pre-Production
On-Set Supervision for Matte Painting Shots
Basic Skills and Tricks of the Trade
Miniatures and Computer-Generated Sets
Finding the Best Frame
Re-Projected Photo Survey
The Need for Creative Compositing
3D Matte Painting
Chapter 7: Digital Element Creation
Digital Modeling
Overview: The Importance of Modeling
Types of Modeling
Model Data Types
Development of Models
Modeling for a Production Pipeline
Engineering Aspects for Polygons
Engineering Aspects for NURBS
Rigging and Animation Rigging
Rigging: What is It?
Animation Rigging
Deformation Rigging
Texturing And Surfacing
The Importance of Texture Painting
Hard Surface Models
Creature Models
Types of Geometry: Their Problems and Benefits
Prepping the Model to Be Painted
Texture Creation
Various Other Map-Driven Effects
Texture Painting in Production
Model Editing
Digital Hair / Fur
Hair Generation Process
General Issues and Solutions
Digital Feathers
Morphology of Real Feathers
Modeling Digital Feathers
Similarities between Hair and Feathers
Differences between Hair and Feathers
General Geometry Instancing
Asset Creation
World Building
Shot Considerations
Dynamics and Simulation
How is a Simulation Created?
When is Simulation Appropriate?
Tricks and Cheats
Important Considerations
Planning and Preparation
Software Solutions: A Broad Overview of Current Options
Particles
What are Particle Systems?
The Next Step
The Birth of Particles
Creating Effects
Rigid-Body Dynamics
How Rigid-Body Dynamics are Created
Potential Problems
Other Issues
Tricks for Getting It Done
Digital Lighting
Light in Reality and in Computer Graphics
Case Study of Reality Compared with Simple CG Simulation
Visual Sophistication through Texture Mapping
Physically Derived Shading Models
Beneath the Surface
Goals of Lighting in Visual Effects
Work Flow for Successful Creative Digital Lighting
The Technologies of Lights in Computer Graphics
Direct Lighting: Source to Surface to Camera
Reflections
Photographed Reflections
Shadows
Image-Based Lighting
Rendering Occlusion
Ambient Occlusion
Reflection Occlusion
Creating Light Sources from Environment Maps
Physically Based Rendering
Physically Plausible Rendering
Volumetric Lighting Effects
Shader Basics
What are Shaders?
Shading Models
Bump and Displacement
Map-Based Shaders
Procedural Shaders
Shader Design
Anti-aliasing Considerations
2D Compositing
2D File Formats
Image Quality: Color Bit Depth and Concatenation
Log vs. Linear
Low Dynamic Range and High Dynamic Range Images
Mattes and Pre-Multiplied Alpha
Working with Rendered CG Elements
Integration Techniques
2D Compositing
Z-Depth Compositing
Adding Depth of Field
Adding Motion Blur
Relighting
3D Compositing
Working with 3D Data in a Compositor
Pan and Tile
Camera Projections
Set Extensions
Coverage Mapping
3D Mattes
3D Retouch and Cleanup
Adjusting Camera Moves
Particles
Deep Compositing
Crowd Generation and Simulation Techniques
Live-Action Replication
Sprites
Computer-Generated Crowds
Modeling for Replication
Variation
Mesh Density
Animation Cycles for Replication
Motion Capture
Keyframe Animation
Dynamic Motion Synthesis
Behaviors and Crowd Control
CG Prosthetics and Actor Enhancements
On-Set Tracking and Capture Considerations
Eye Enhancements
3D Techniques
2D Techniques
2D Techniques
Silhouette Changes
Re-Projection
3D Products, Systems, and Software
Digital Element Creation Process
3D Graphics Software
3D Tracking
Special Effects
Rendering
Texturing
Chapter 8: Interactive Games
Overview
How the Gaming Industry and Film/TV Industries are different
Game Engines and Real-Time Rendering
Runtime Component
Disciplines and Job Titles
Game Design
Engineering
Production
Test
Art
The Art Director
Concept Art
Environment Artists
Texture Artist
Characters
Hard Modeling
Props
Lighting
Baked vs. Dynamic Lighting
Shadows
Effects
System Effects
Environmental FX
Breakables
Destruction
Tech Artist
Animation
UI
Real Time Shaders and Materials
Pre-Rendered Cinematics vs. Real Time Visuals
Optimization and Runtime Budgets
Performance Analysis and Profiling
CPU vs. GPU bound
Technical Terminology
User Calibration
Latency
"Game Mode" on Televisions
HDR10
PBR – Physically Based Rendering
FBX
Mip Mapping
Filtering
Texel
Screen Space Ambient Occlusion
Level of Detail
Vertex Shaders and Fragment Shaders
AR/VR
Future of Gaming
On-Demand Rendering, Cloud Distribution and Ray Tracing
Chapter 9: Complete Animation
What Is An Animation Project?
Full Animation versus Visual Effects
Difference Between Visual Effects and Animation
Production Pipelines
Production
A Survey and History of Animation Techniques
Traditional Animation
Stop-Motion
Computer Graphic Technology
Considerations for a Full CG-Animated Feature Pipeline
CG Feature Animation Pipeline
Managing an Animated Film
Film Management and Personal Style
Building Brain Trusts
Building the Core Creative Team
Writing and Visual Development
Working with a Studio
Facilities and Environment
Managing the Event
The Production Process: An Animator’s Perspective
Working on CG-Animated Content in Live-Action Features
Planning the Process
Production
Character and Environment Interaction
Chapter 10: General Workflow Considerations
Virtual Studio Technology
Analysis of A Production Workflow
From Workflow to Pipeline
Service Bureau versus In-House Requirements
Design of a Production Workflow
From Analysis to Design
Deploying a Production Workflow
From Design to Implementation
Infrastructure
Tracking Assets
What is Task and Asset Tracking?
Commercial Task and Asset Tracking Systems
Building Task and Asset Tracking Systems
Scene Assembly
3D Scene Assembly
2D Scene Assembly (Compositing)
Working Across Multiple Facilities
Images
Models
Texturing
Animation
Compositing
R&D
Chapter 11: VR / AR (Virtual / Augmented Reality)
A Note from the Editors
Prelude to Virtual Reality & Augmented Reality
Pre-Production for VR/AR
Production for VR/AR
Post-Production for VR/AR
Editorial Post-Production for VR/AR
How to Direct the Viewer?
The Post-Process
Nonlinear Editorial, Timelines and Edits
World Lock and Forced Perspective
Types of Head-Mounted and Handheld Displays
What are the HMD and the "VR Presence"?
Electronical Designs of HMDs and their Respective Performances
Optical Designs of HMDs and their Respective Usages
Image Quality Factors in VR HMD Displays
Hemispheres and Domes
Game Mechanics Are What it is All About
Overcoming Doubt and Preconceived Notions
Dome Projections
The Future of Domes
VR/AR Tracking Displays
Uses of VR
Enterprise
Narrative Storytelling
Future of VR and AR
This is Just the Beginning
Acknowledgments
Appendix A: Charts and Formulas
Appendix B: Credits / Titles to Be Submitted in Accordance with VES Guidelines
Appendix C: Glossary
Index
