FDT interview about ALEXA SUP 9.0
In the February issue of his popular industry journal Film and Digital Times, Jon Fauer, ASC, interviews Marc Shipman-Mueller, ARRI Product Manager of Camera Systems, and Henning Rädlein, Head of ARRI Digital Workflow Solutions. They speak about ALEXA's new Software Update Packet (SUP) 9.0 and how it fits into current production and postproduction procedures. A few additional explanatory notes about Codex technology and files [in brackets] come from Sarah Priestnall of Codex. The interview was first published in Film and Digital Times, issue 58-59.
Marc and Henning, what's new in ARRI Software Update Packet (SUP) 9.0?
Marc: Let's begin with support for ProRes. With SUP 9.0, we can now record with the XT camera, ProRes 4444 (4x4) up to 120 fps. Previously ProRes 4x4 only went to 60 frames a second. With SUP 9.0 on the XT cameras, it's up to 120 frames a second as long as you record to XR Capture Drives or CFast 2.0 cards. Another update is that we now have DNxHD on all cameras, Classic ALEXAs as well as ALEXA XTs, and we've added DNxHD 444.
I think a large portion of customers record on ProRes 4444 Log C and I always say if we would have known this we probably could have avoided doing some of the other flavors of ProRes. So much stuff is done 4x4 Log C. That is the standard. What we saw at the beginning of 2013 was an increase in ARRIRAW productions, but people didn't like the fact that they needed to use an extra recorder on top of the ALEXA. When the ALEXA XT came out a lot of people who wanted to do ARRIRAW got the chance to record it internally. We're seeing ARRIRAW on some of the not-so-high budget feature films now, as well as on high-end commercials; because with the ALEXA XT camera, ARRIRAW recording is all included in the camera.
With SUP 9.0 supporting the CFast 2.0 card, are you seeing more people using them on ALEXAs?
Marc: That's starting very slowly. The cards are just coming to the market. I think we've just received our first adapters from Codex for the CFast 2.0 cards. They are the same cards that the AMIRA uses. I actually think very little is going to happen on that front with ALEXA until the AMIRA hits and then some people with both cameras are going to choose to buy CFast cards. Right now the advantage of CFast cards on ALEXA is recording ProRes 4x4 at 120 fps, whereas on the SxS Pro 64 GB card you can only record up to 60 fps.
Let's talk about the ALEXA XT Open Gate in SUP 9.0.
Marc: Open Gate works only for ALEXA XT, XT Plus and XT Studio cameras in ARRIRAW. ALEXA XT M support is planned for Software Update Packet 10, which, by the way, we're working on right now. Open Gate on the XT cameras uses the sensor's entire active image area of 3414 x 2198 photosites.
We think that most people will continue to shoot most of their footage in 16:9 for spherical or 4:3 for anamorphic shows. I don't think people are going to shoot a whole show on Open Gate, even though it's an interesting industry and anything is possible. I think Open Gate will be used for visual effects, wide angle establishing shots, things like that. They may record those scenes in Open Gate, and then switch back to whatever their regular format is for the rest of the show.
How do you switch from regular to Open Gate if you are a camera crew in the field? Do you have to send it back to ARRI?
Marc: It depends what kind of camera you have. If you an ALEXA XT made before December 1st, 2013, the camera actually has to go into ARRI service for a special adjustment. You have to have Software Update Packet 9.0 installed. If you have an ALEXA XT that was built after December 1, you just have to make sure it has SUP 9.0 and then there's just one switch in the menu. You select from sensor mode 16:9, 4:3, or Open Gate. It takes about 30 to 40 seconds and then you are in Open Gate and you can record ARRIRAW. One thing is important to note. Since we are using the whole sensor, you now have an image circle of about 33.5 mm. So you must be sure to test, especially with wide angle lenses, to make sure that they cover the entire image area. We found with our wide angle Master Primes and Ultra Primes the boundary is somewhere around 18 to 21 mm. Our 9.5-18 mm Ultra Wide Angle Zoom covers fully. With other zoom lenses, it's all over the place, so be careful.
In postproduction, how do you know that it's Open Gate? Is it in metadata?
Henning: Yes, it's in metadata. We are currently updating all the SDKs out there and informing all our partner companies about this new format. The ARRIRAW converter is available and able to process it. Very soon all the tools will be able to understand automatically that this is another format. They can downscale, or we'll offer options in another menu. I think most people will process without downscaling. They'll do their effects work or pan-scan, and then downscale to the HD or 2K deliverable format or upscale to 4K or UHD-1.
What does Open Gate look like in dailies? Do you see it in a new aspect ratio with black on the sides?
Henning: You will get a pillarbox image if you do 16:9 offline editing which is probably the preferred way. So you will have right and left black bars if you want to see the full height of the image. The important thing is that you see all that has been shot. The aspect ratio of Open Gate is 1.55:1.
At what stage does an editor do the repositioning of an Open Gate image?
Henning: I talked earlier today with somebody in India who said they want to shoot the whole film in Open Gate. I also hear that several studios in the US are planning to do so. In this case, they will probably downsize to the final 'Scope or 1.85 ratio for editing in dailies and pull the original Open Gate to discuss VFX work. It's a different situation for each production.
Marc: We have another guy who wants to shoot in the Himalayas right now -- not a very easy location -- they're thinking Open Gate, wide landscape, big sky. As I said, it's an interesting industry and sometimes we are surprised how far filmmakers go with the tools we provide.
Henning: When I see the superb quality of ARRIRAW I would stay with that pixel count for most of the production. You can go with the Open Gate format for those shots that really make sense. This is definitely a handmade visual effects scenario. Of course, you have to reposition and alter the frame lines.
Marc: There are user rectangles you can set. We have frame lines for most of the common formats that are built in. The one thing you can't do is set user frame lines. Those are custom lines you can make. Open Gate doesn't do audio. It's not quite as fast as 4:3 in terms of maximum frame rate. So there are some disadvantages you inherit when you do this.
I've talked to a number of people who said, "You know what, I don't necessarily want 4K. I'd just like a little more, a few more pixels for repositioning, rotating, stabilizing, that kind of stuff." Those people are mostly car shooters. I think to them that would be a great option because you don't incur the data weight penalty you have with 4K. You can still shoot uncompressed, unencrypted ARRIRAW and have a few more pixels around the edges.
Explain the new pre-recording feature of SUP 9.0.
Marc: Pre-recording for ProRes essentially uses the storage space on the media: SxS Pro, CFast 2.0 or XR Capture Drives as a ring buffer. When you go to prerecording mode the camera will continuously record the footage and store between 8 and 60 seconds, depending on your frame rate and codec. When you push the record button it'll actually keep a certain number of seconds before you started recording. So you can capture an event that happened before you push the record button.
Before you get eaten by the tiger that's about to charge at you when you're doing wildlife photography.
Marc: You know, I was talking to a nature cinematographer who spends most of his time filming owls. They will sit for hours on a branch and then all of a sudden take off -- the owls, not the cinematographers. If you don't push the button at just the right time you miss it and the whole day is wasted. So if you give him the option to be able to capture the 10 seconds before the owl takes off, his day would be saved. That's a new feature of SUP 9.0 and that works for all the ALEXAs: Classic as well as XT cameras, and Classic cameras upgraded with the XR module.
Henning: Is it actually buffering in the camera or on the card?
Marc: It's buffering on the recording media. We didn't have enough storage in the camera so we're buffering out to the recording media, which has an advantage. If you use a slim codec you can buffer up to a minute--much more than most other cameras.
What else is new on SUP 9.0?
Marc: We have extra features for the Wireless Control Unit, WCU-4, so you can see what the pre- recording status is. You can switch from regular to high speed. There are some new motor functions. And we have just introduced a new user button, the PHASE button, which I'm sure you remember from the film cameras, where you push the PHASE button to remove the television scan line from the picture. Not a lot of people are shooting with CRTs or real projectors, but there still are some.
There's one more thing that I think is important: protecting data. One of our philosophies is that the image is holy. We always need to make sure that you can record a good image. In fact, most of our software is in layers and the holy layer is concerned with recording the image. Accessories and other things are in an outer layer, so if anything goes wrong you're still going to be able to record an image. We protect your images. So if you examine the progression of our ProRes recording, here is what happens: the camera opens the QuickTime wrapper, starts recording footage into it, and every second it closes the wrapper. Then it opens it again, ads another second of footage, and closes it again. So if your battery goes down--you experience a power loss. If the wrapper were still open while the power was lost, everything recorded into the open wrapper would be lost. But with our system, you just lose the last second because the previous wrapper was closed.
Now this works really well and we've had that all along. But then we found that the metadata sometimes got scrambled when we had a power loss. Your footage was still OK, but the camera wouldn't allow you to record onto the card anymore because it couldn't make sense of the metadata. With SUP 9.0, if the camera finds a card with scrambled metadata it'll look at the actual footage, analyze the footage, look at the metadata and then rewrite the metadata based on the footage so you can still continue to record on that card.
Henning: We have a good name for that: Self-Healing Metadata.
Let's talk about metadata in 9.0--LDS and /i support.
Marc: We now have built-in /i system support for all the ALEXA cameras. If you put a Cooke /i lens or an Angenieux /i lens on the ALEXA it will be able to read that data through the lens connector and store the metadata. Our philosophy with the ALEXA is we're happy to take all the metadata we can get, and put it everywhere we can. So the ARRIRAW has the metadata. QuickTime ProRes and MXF DNxHD contain metadata. The HD-SDI stream contains metadata on the Rec-Out, as well as the Monitor Out. There's an application called Lightcraft where they take the ALEXA metadata that's included in the HD-SDI feed and they are processing it in real time. I know that Transvideo gets metadata out of the HD-SDI stream and they are displaying it on their monitors, along with graphical depth of field read-outs. Metadata has always been a chicken and an egg situation, as you know. You've talked about metadata for decades. We thought if we provided it as widely as possible, somebody will find an application for it. And we're seeing that slowly happening.
How does the Codex Data Logger One external metadata recorder attach?
Henning: Codex Data Logger One is aiming for people who are not delivering their metadata through the camera. They are thinking about this as a postproduction center along with the Vault that's getting the image and the data from other lenses. You can marry everything in the Vault and prepare that for further deliverables in editing and copies and proxies and also export it into databases. So Codex is more system driven, able to handle a huge amount of data and also different camera types. [The Codex Data Logger One also captures additional information such as inertial data and GPS. It supports /i Squared, is integrated with other devices such as Preston and provides consistency across cameras and lenses.]
Marc: You might not need that with an ALEXA because data is integrated on the ALEXA. But let's say, gasp, you use a non-ALEXA camera. You could use the Data Logger box with a camera that doesn't have the ability to capture the lens data inside the camera. With the ALEXA we're trying to integrate everything. The LDS capture is integrated in the camera. We're capturing all the camera information. And with the Plus camera, even the remote control ability is built into the camera so you don't have to attach extra boxes. With other cameras you may need an external box to record the lens data and then you need another external box for wireless follow focus. With the ALEXA XT Plus, for instance, all that is already integrated into the camera so you have fewer boxes, fewer cables and fewer breakdowns.
If you're archiving to LTO does the metadata go along with it as well?
Henning: Yes. With ARRIRAW, the metadata is in the ARRIRAW header, and it stays there. Last week we published the ARRIRAW DPX multi chunk header description [which was developed by Codex]. Before that, DPX files were a big problem because, although they were a standard in the industry, there was no standard to put metadata in DPX. We created a chunk header which actually copies the entire ARRI header into the DPX file and makes a description so that, for example, The Foundry (Nuke) and other companies can use it and display the metadata even if the ARRIRAW was converted to DPX by another vendor. If somebody changed the metadata, we can see the original and what was changed. Keep it transparent!
For example, let's say you're working with the ARRIRAW file converter and you see a file that was not processed with the 800 ASA setting but with the 200 setting for whatever reason. Or someone changed the white balance. The operator can see something is wrong. There was no system with that luxury for the VFX world up to now.
I talked to the effects supervisor on LIFE OF PI and he told me how Claudio Miranda, ASC, the DP, might be looking for a scene shot with a 25mm Master Prime. So they looked in their database for all the occurrences of 25 mm Master Prime shots and they found it very quickly. This is a tool that can really help the visual effects people -- an industry not making enough money. LDS and intelligent handling of metadata is something that can help them. It is a boring issue and it is not fancy or sexy, but if you use it intelligently, you can save a lot of money, time and effort in searching and setup.
You both know I'm a huge advocate of metadata, but it seems like few are using it to its proper advantage.
Marc: It's coming. Here's a lovely, recent example. I visited the folks shooting GAME OF THRONES in Ireland. They're very interesting because they're HBO. They're not beholden to any regular studio so they kind of came up with their own procedures. They're using the metadata a lot. They're shooting with ALEXA onto Codex Recorders. When a digital magazine comes off the Codex they put it in a special Codex Onboard M in the post suite that has its recording disabled. They go through it and they make sure all the metadata is correct. They've found that if any metadata right at the beginning is wrong, it will propagate throughout all the deliverables, all the proxies, and then you can never fix it, never get it right. So before they do anything else, even before they do a backup, they check metadata. I've never seen this before--actually making sure the metadata is correct as the very first step.
[Because they use Codex, they can take advantage of the dynamic metadata capability of the Codex Virtual File System (VFS). The VFS is a part of all Codex products: one of its benefits is that metadata can be viewed and edited.]
Then they do the backups with the correct metadata, do the dailies, and then everything else follows from there. This shows how important metadata is to their workflow.
Many productions are editing with Avid, conforming ARRIRAW as DPX files and then color correcting DPX files. Are you using DPX files as well at ARRI Film & TV?
Henning: We're doing DPX also. Sometimes clients have gone to Open EXR -- more visual effects driven companies. We have to say that we get very, very good color correction results from the DPX format, which means that one does the ARRIRAW processing before conform and grading. So there's actually no need to debayer live during color grading. You can do that, but you get the same result, in our opinion, when you first convert to a DPX file and then color correct from there--because you're ending up both ways in the same Log C wide gamut color space and grade based on this, hopefully using a Look Up Table.
But you're throwing out metadata at that color grading stage?
Henning: When we have integrated output to DPX with metadata it would not be lost. The most important thing is that the processing is done with the right white balance and a right exposure index. This is directly out of the ARRIRAW file. Then you go into the DPX file which only contains the preprocessed image for the color timing.
Wouldn't it be good for metadata to go all the way through to grading? The DP is in the DI grading suite and can't find the meticulous notes jotted down for Scene 105, Take 6. It was supposed to be a little warmer with a little help to pull down the sky. The camera reports can't be found.
Marc: I think you're right. However, I see a new trend now, which is to do color on the set. I traveled the U.S. this year and went to a number of sets, including X-MEN and GODZILLA. On all these shows the DIT was acting like a colorist on the set. The DP would constantly tweak the image to get essentially a color corrected image on the set, or at least a preview color correction. The CDL would then go with the footage for the dailies creation, and would be applied so that the dailies had the same color correction as they were seeing on the set. They would also sometimes use that CDL as a basis for the final grading. I've seen this in two versions. Version one is you just do the primaries on the set; that goes into CDL, travels to dailies, travels to the final grading. And there's a second version, which I've seen on some of the high-end feature films. The DP would go into a special trailer at the end of the day to do color grading with primaries, secondaries and power windows, even render out the DPXs so the special effects people would get proxy DPXs that already have the color timing the DP wants. When they do their special effects they already know what the end result really is going to be. I've heard a lot of DPs say that they're not invited to final color grading sessions or they don't have enough time and so they're trying to do as much as they can on the set now.
That's a very good point, especially if you're not invited or don't have time or are working on another show.
Marc: The result is they're grading on the set now. There's so much of it that the DIT has become the on-set grader for a lot of people. We're supporting that with the XT camera in that you can actually load the CDL from the DIT's laptop via an Ethernet connection and attach it to the ARRIRAW file.
How is that attached, as metadata or...
Henning: We have some space free in the metadata header. We gave the ARRI header specifications to all the partners, including Codex, and they write the information into the ARRIRAW file which goes all the way through until the end.
Is this only possible if you're using a Vault or can it be done with anything?
Henning: You have to use the Codex virtual file system. It's not about the Vault hardware box. It is about the Codex virtual file system [which is an integral part of all Codex products].
Marc: It works with the Vault, with the single dock if you have a license for the Codex virtual file system, and it works with a dual dock. Those are the three download hardware platforms that would support that.
What are the most DITs using for grading on set or near set?
Henning: In my opinion, most of them are using Resolve. It's a very good system. It's cost-free. It's an incredible tool. The Lite version can be downloaded for free. I looked it up today and it does almost the same as the big version. It doesn't have support for de-noising, stereo 3D, or the big panel, and is limited to UHD. Many of the big guys are using Colorfront OSD. It is a very common tool, although the strength of Colorfront is actually to make the deliverables and dailies.
Marc: On X-MEN they were using EFilm Colorstream software. Other jobs had Pomfort LiveGrade. The hardware all the DITs seem to be using is the Blackmagic Design HD-link. Everybody uses that hardware, everybody. There's different software being used, but everybody has the Blackmagic Design HD Link.
Henning: It's a box and it applies the 3D Look Up Table they have just generated so that it can be shown on the monitors around the set and in the video villages.
It's true, every job is different.
Henning: You know, this is the problem with workflows. If you write a paper and give some workflow recommendations, they are probably wrong because somebody is already working on another system. Or they are seeing everything in P3 or they're talking about ACES or they have to deliver visual effects pulls to a company that wants it in ACES and Open EXR. There are so many different configurations worldwide that now our thinking is that it's better to describe the single components. Do not tell them they have to do it this way or that.
Where do you do the software?
Marc: It's all German software. A lot is done right here in Munich. A lot of our camera is software, but the thing that really surprised me is how much we have to do in testing. With our film cameras we had a testing department with maybe three or four people. We now have a huge testing procedure that goes in steps. When a piece of software is written it goes first to the research and development internal testing department. They do the first testing. If anything is wrong, it goes back to the programmers. Then back to the R&D internal testing department, until they find it OK. Then it goes to the second stage testing, with some people in Henning's group who are more practical and try to break it. They're very good at it.
Henning: Yes, we are good. R&D hates us.
Marc: When they find bugs, it goes back to the programmers. When Henning's people think it's okay then it goes to the quality control department who do their own round of testing and look at everything. In reality, some of this happens in parallel, but all software goes through the whole process.
A camera cannot have a software crash on set or on location. That's why you don't want to cut any corners and all these testers are there to make sure. It's tedious. You change one thing, you have to retest everything over and over again.
In Europe, what are people using mostly for editing?
Henning: It depends on the market. Commercials and broadcast are very much Avid driven. Adobe is getting stronger, mainly in the broadcast industry. Documentaries and low budget productions are using Adobe Premiere and Final Cut Pro.
Marc: I think editing in Hollywood and on high-end features happens in Avid, but that's mostly as proxies. The idea of mastering onto a compressed format really started with ProRes. It is slowly starting to take a foothold with DNxHD.
Are they mastering in DNxHD?
Marc: There are some productions mastering in DNxHD, but it's a slow start. We were actually surprised. We thought there would be many more people doing this. Now in SUP 9.0 we have DNxHD 444, which is the highest quality DNx, we think there will be more people mastering that way.
You think they'll be mastering on the Avid or are they still going to finish traditionally?
Henning: I think most are finishing traditionally. But we would like to see DNx productions shooting in Log C. DNxHD 444 is a big improvement and quite equal to ProRes 4x4. It also has the possibility of providing very good keying capabilities. It's 10-bit and the data rate is high.
What are most of the high-end jobs, commercials and features, finishing on right now?
Marc: Most high-end commercials and feature films are going ARRIRAW; the rest are ProRes 4x4 (ProRes 4444) Log C.
How are they finishing the ARRIRAW jobs?
Henning: DaVinci Resolve is getting stronger and stronger. Many conform and color grade in Resolve. It's actually ARRIRAW certified now since Version 9.1 and Version 10 and has a very good image quality. We like it very much and think it has the same debayer quality as the ARRIRAW converter.
A lot of people are using Resolve for ARRIRAW finishing, but of course, this depends on what the postproduction company has installed. The ARRIRAW SDK is also used on Autodesk Smoke, Flame, Lustre, Scratch, Clipster, Nucoda, and Pablo to easily process ARRIRAW in the best quality. Also, many use the Codex debayer or finish in Baselight. Those tools are also ARRIRAW certified.
Marc: SDK (Software Developers Kit) means they have our debayering engine and put it in their product. Resolve has written their own debayering engine which we consider on par with ours.
What are you using at ARRI Film & TV Services?
Henning: At ARRI Film & TV it's historically like a mixed drugstore. A while ago, I bought Digital Vision Nucoda systems because I felt that traditional color timers coming from the film lab would feel comfortable with their system. It was the easiest to understand. Lustre was, at that time, very expensive. However, we had the first Lustre in Germany and still have two, but they were more complicated and harder to understand. Commercial people like Baselight very much, so we got those as well. So it's a mixed bag. Then our workflow crew and I brought in Resolve and built up a good relationship with Blackmagic Design.
What are you using for online mostly?
Henning: When we do camera tests and that kind of stuff, we're doing that all on Resolve.
You're doing the assembly and the color grading on Resolve?
Henning: Yes. You know, it's kind of an automatic process. The assembly takes about the same amount of time for me to get a quick coffee and then it's done. It is always nice to leave the dark room.
Are more people just buying new ALEXA XT cameras, rather than upgrading to the XR?
Marc: More than we thought. At first, more people upgraded. But now we see it's the other way around. More people are buying the new ALEXA XT cameras. The really interesting thing is they are not selling their old cameras, as we thought they would. A lot of rental houses buy the XT cameras on top of the existing Classic cameras. They want to use the ALEXA Classic camera for lower budget shows. And the XT cameras are for those productions that demand the latest equipment. I also think what's happened is that unfortunately shooting film has dropped off. So all those people in the 30% of the market that still had been shooting film now also need digital cameras. I think there is a demand for more digital cameras to come to the market to replace the film cameras. The XT cameras have been a huge success.
And a lot of productions are shooting with more than just one camera. They are shooting with multiple cameras for every scene. So you'll see features with 3 to 9 cameras on many setups.
Henning: Pity the editors.
But good for you, selling a lot of cameras.
Marc: I was recently updating the website. We now have 11 cameras with 3 licenses and 5 different modes, sensor sizes, high speed, low speed -- so we need really good overview charts.
Henning: Our online Tech Talks are little stories of the week--short tutorial films, where Marc and Hendrik and others explain products. Hendrik is our product manager for wireless accessories. We have also posted 50 short videos explaining the ARRIRAW converter. We discussed writing a manual, but we decided to do these short 2½ minute videos instead (see link at end of article). We can guide people directly to a link to a short video that answers their question. For example, "How do I extract metadata from an ARRIRAW file?" I send them a link, they spend 2½ minutes watching and then they know.
What do you have in store for us for Software 10?
Marc: Well, we're still looking at that. There is still some haggling going on. I have a huge list of over 500 feature requests. They keep coming in and that is good. Many customers send me ideas, or somebody from the testing department comes to me and says, "Marc, we could put this menu item here and do this and it'll all be a little easier."
So I collect all these suggestions and I think it's great that people make all these suggestions. Whenever we have a new software update we clearly have a limited budget and I'll try to see what the most important thing is to put into the new software upgrade. Then we assess how much it will cost, what the risk is, who has to work on it, and how long it'll take, and then we make choices. Right now we are in the process of that and we'll probably have a finished plan for SUP 10 in early 2014.
We've gotten feedback from a number of people saying it's good to see that ARRI continues to provide significant features. We love our cameras and want to make them as good as they can be. And from a slightly less romantic angle I think that makes the business model represented by ALEXA something that really works: long product cycles.