Color image encodings and metadata

The post-production workflow
ACES post-production emphasizes consistency of color in four contexts:

  • on-set monitoring
  • dailies viewing, whether in a director’s screening room or on a producer’s tablet
  • visual effects development and review
  • final DI grading

Providing consistent color across a variety of device types, file formats, and viewing environments requires considerable behind-the-scenes mechanism to make things "just work". This is true whether or not one is using ACES. The goal of ACES-based workflows is to enable flexibility while keeping the fundamental building blocks and pipeline consistent.

Color spaces and color encodings
Post-production in ACES introduced an additional color space for visual effects work named ACEScg. Like ACES itself, this is a linear color space. But where ACES uses a set of color primaries (termed "AP0") that encompass all visual colors, ACEScg uses the same set of primaries used by ACESproxy and ACEScc — a set of primaries named "AP1". AP1 primaries are nearer the spectral locus, and are closer to traditional grading primaries than are the AP0 primaries. The relationship between the color primaries of ALEXA wide-gamut, of ACES, and of ACESproxy/ACEScc/ACEScg is shown in the figure below.

Recapitulating all the color spaces involved:

  • ARRI V3 Log C: uses the ALEXA wide-gamut [AWG] primaries, and a log floating-point or log 10-bit or 12-bit integer encoding. An image storage space and a grading space.
  • ACES: AP0 primaries; linear floating-point encoding. An image storage and interchange space.
  • ACESproxy: AP1 primaries, log 10-bit or 12-bit integer encoding. A grading space. Not an image storage space because of insufficient bit depth to guarantee lack of banding.
  • ACEScc: AP1 primaries, log floating-point encoding. A grading space. Not an image storage space.
  • ACEScg: AP1 primaries, linear floating-point encoding. A working space for rendering and compositing. Potentially an image storage space, but not a recommended image interchange space.

Note that the log encoding for ACESproxy and the log encoding for ACEScc were designed to work together. If ASC CDL values were created using an ACES-supporting on-set tool that had applied those values to ACESproxy images, those same ASC CDL values will produce the same visual result when applied in downstream ACES-supporting tools that manipulate ACEScc images.

Flow of color image data and metadata across contexts
There are two key principles that provide consistent color across the four contexts mentioned above:

  • The same core set of transforms is used at every stage
  • If a transform is parameterized (in the way that an ASC CDL transform is parameterized by slope, offset, power and saturation values, or a 3D LUT transform is parameterized by the contents of a particular 3D LUT) then the same parameters are passed from one context to another throughout production and post-production

On-set production optionally produces an on-set grade. If it does produce such a grade, then the grading parameters, expressed as an ASC CDL file, are pased from on-set to near-set or Editorial, and thence to VFX, and finally (as a starting point or hint) to the final DI grade.

Similarly, any 3D LUT that might be implementing an overall show look is applied the same way, at the same place in the pipeline, in each context.

The diagram below shows the flow of image data (and, with dashed lines, of optional image metadata) between production and post-production contexts.

On-set has been covered in the "Recording" tab, and near-set or Editorial has been covered in the "Dailies" tab. This tab therefore focuses on the two contexts most tied to post-production: Visual Effects and final DI grading.

Visual effects

Using current software
ACES imagery requires a different set of image colorspace conversion and image viewing transforms than those to which users of vendor-specific workflows may have become accustomed. Fortunately, many visual effects tools today integrate OpenColorIO, a flexible open-source package for color management. Versions of OpenColorIO supporting ACES Release 1.0 became available in early 2015.

VFX tools like Nuke or Fusion rely on OpenColorIO (OCIO) for color management. When the latest versions of OCIO and its configuration files (ACES 1.0.1 or later) are used, these tools are ACES 1.0-compliant.

Nuke 10 has enhanced its support for OCIO, including incorporation of ACES 1.0.1-compliant OCIO configuration files. The related Project Settings panel choices have changed slightly, as shown below:

Note the use of the Nuke_1.0.1 OCIO configuration, bundled with Nuke 10.0 and later versions. Also note that for this project the default for “float files” has been changed to be ACES 2065-1; in current distributions shipped by The Foundry, this (somewhat controversially) does not reflect the original intent of the Academy, that images written to disk be expressed in AP0 primaries, not AP1 primaries.

Obtaining ACES plates
Background plates may be delivered to VFX as ACES Container files (i.e. OpenEXR files containing ACES data), with the conversion of ARRI camera output to ACES happening upstream of the VFX facility as part of the "pull". Alternatively, the conversion can be done at the VFX facility, either as a one-time batch process, or as part of the composite itself.

Batch conversion to OpenEXR has been covered in the "Dailies" tab. In the image below, we give an example of the Nuke Read node settings required to convert an ARRIRAW file to ACES.

Here the salient points are that the "color space" pop-up in the lower "ari Options" part of the Read node’s properties panel has been set to "Scene Lin. - ACES" and that the colorspace used for linearization in the upper part of the panel has been set to match, with a value of "ACES - ACES2065-1".

Note that the version of the ARRIRAW SDK used by Nuke does not offer the newer ACEScc or ACEScg color spaces as ARRIRAW decoder output options, so the label it provides to the Nuke UI code ("Scene Lin. - ACES") is effectively a shorthand for what Nuke more precisely identifies as "ACES - ACES2065-1".

In Nuke 10, reading in DPX files, TIFF files, ProRes clips or DNxHD clips containing Log C images to be brought into ACES containing Log C imagery is straightforward. The color space of the image data read from the files or clip is indicated in the Read node (including the Log C image’s exposure index), and Nuke converts the newly-read imagery from the indicated color space to the working space established in the project settings.

Note that the situation is slightly more complicated when using Read nodes in prior versions of Nuke. In the older versions of Nuke, there were two nodes required: a read node with "colorspace" set to "linear" and an OCIOColorSpace node converting from the EI-specific ARRI V3 Log C version to the working color space for the production.

The VFX "neutral grade"
When a squence is comprised of many shots, and each shot has multiple takes, it is common for the scene illumination of on-location shots to drift. The color temperature and intensity of natural light changes over time, and the plates for a complicated VFX sequence may take many hours (if not days) to shoot.

Especially when physically-based modeling and rendering are used, it is more efficient to have a compositing supervisor normalize all the shots in the sequence to some "neutral grade" before distributing the work amongst individual artists. This type of grade is always performed using solely linear operations such as per-channel gain changes, or 3x3 matrix multiplications.

Viewing transformations
OCIO provides for viewer process application of the ACES output transform (the Reference Rendering Transform [RRT] and some Output Device Transform [ODT]). In OCIO-color-managed Nuke, this is controlled by a drop-down menu in the main menu bar. In the illustration below, a Rec. 709 ODT has been selected from the OCIO’s pre-packaged library of combinations of RRT and various ODTs.

Matching on-set graded color at the artist’s desktop
Changes to the default rendering transform that were effected by the use of on-set ASC CDL operations on ACESproxy data can be mimicked at the artist’s desktop, by applying the ASC CDL operations to ACEScc data. Because most compositing programs work in linear space, before the ASC CDL is applied there will need to be a transformation between color spaces performed between the working space and ACEScc, and then after the ASC CDL is applied there will need to be a transformation from ACEScc back to the linear working space.

The following diagram shows the flow of image data and look metadata from the set, through editorial, into VFX. It does not show flow of image data and look metadata out of VFX and into final DI grading, other than to have solid or dashed arrows showing that the flow of data continues past this step.

Download PDF

In Nuke, with the ACES OCIO configuration active, the working space is ACEScg, and the viewer process expects ACEScg input. So at the point in the VFX section of the diagram just prior to the application of the On-set grade, where a two-step conversion takes place from ACEScg to ACES and then from ACES to ACEScc, an actual compositing script can collapse these two nodes into a single ACEScg to ACEScc OCIO Colorspace node. In addition, as the working space of the ACES OCIO configuration is ACEScg, the ACEScc to ACES transition immediately following in the diagram would instead be an ACEScc to ACEScg transition.

In fact, it is possible to collapse these three nodes even further, by noting that we are transforming from ACEScg to ACEScc, then applying the CDL, then transforming from ACEScc to ACEScg. This sort of ‘wrapping’ transform is exactly what the ‘working space’ colorspace selection in the OCIO CDLTransform node does:

VFX deliverables
The deliverables from a VFX workflow should be OpenEXR files (specifically, ACES Container files), with any LMTs used in viewing the composite accompanying the delivered frames (or being incorporated by an included reference). The effects of the LMT(s) should not be "baked in" to the resulting OpenEXR frames; this includes both the LMT effecting the on-set grade as well as the LMT(s) effecting any show look. Keeping the on-set grade and the look out of the image data, while passing on the on-set grade and the look as image metadata, allows the production’s creative staff to make straightforward and consistent global changes, if need be, in the final DI grade.

Final DI grading

Image and color correction color spaces
All major color grading systems today are capable of reading ACES container files, or of reading files created directly by a digital camera and applying the appropriate IDT or to produce ACES image data. Alternatively, ACES provides for a special type of film scan — an "ADX" scan — and transforming the scanned data into ACES imagery using a special film-to-digital transform termed the "unbuild".

Color-correcting an ACES image, whether it was born of a digital or of a film camera, is not very pleasant for the colorist, because ACES images are linear, and the human visual system’s response is logarithmic. So, just as when applying ASC CDL values, final DI grading is done in a logarithmic space. This is the ACEScc space, designed so that ASC CSL color manipulations made on-set to ACESproxy values will have the same effect if applied in the final DI grading session to ACEScc values.

This is shown in the diagram below, which is an extended version of the previous diagram that showed flow of image data and metadata through VFX. In this diagram, the flow is extended to cover final grading and creation of deliverables as well.

Download PDF

Note that though on-set grading information continues to accompany the image data, at this point it is more in the nature of a reference, or a hint, or a starting point. The entire point of final DI grading is to refine the look of the production, so the equality of displayed image that was present between on-set and editorial, and between on-set and editorial and VFX, will probably not be extended to the final DI grade.