Blu-ray 3D Conversion Virtual Roundtable
This morning I sat in on a virtual roundtable hosted by the Blu-ray Disc Association’s chairperson Andy Parsons. He sat down with Ian Harvey who is the Senior Vice President, Advanced Technology for Twentieth Century Fox Home Entertainment about the technology behind converting a film to Blu-ray 3D. The film in question was Fox’s recently re-released i,Robot. Below is a transcript of the chat. Well, most of the chat. I forgot to turn on my recorder and missed most of Parsons’s opening greeting. Also, I took some screencaps so you could figure out what they’re talking about, but a few of the things he’s talking about I didn’t manage to cap, so it might be a little confusing. Whoops.
Andy Parsons: Good 3D is just amazing. So, Ian, can you give us a quick idea of what’s involved in 3D and how we make it work?
Ian Harvey: Sure, sure. Fundamentally, we have to take a 2D image – this is a 2D image of i,Robot – and we have to create two views of that, slightly different from each other, so that your right eye sees something different than your left eye. Just like in the real world, you have to see that difference between your two eyes for your brain to comprehend the 3D. This is a flat screen. There is no 3D associated with it. So we have to convince your brain that there’s actually a difference between the two images. So when you’re shooting it with a camera, that’s fine, you can capture the two images. However, when you are converting the 2D, you have to take the 2D image, bring out the points that you want to have in the three dimensions, and create a view for the left eye and a view for the right eye. Clearly, that’s been done with the red/green glasses, everybody has seen those. There are polarized glasses, more commonly used in theaters. The best case we’re looking at today, we’re looking at a Panasonic monitor with active glasses. So, shutter glasses, synchronized full screen left eye with the left eye lens open, full screen right eye with the right eye lens open.
AP: This is happening so fast you can’t really perceive it, right?
IH: Correct. It’s so fast that your brain doesn’t perceive that there’s actually two images. Your brain thinks you see one. Your left eye sees something slightly different than your right, therefore you can see the 3D.
AP: I see. So it seems like you take something that was originally shot in 2D and now you want to convert it into very convincing 3D; that must be a pretty complicated process? Is that a fair statement?
IH: Yeah, it’s a fair statement. There’s varying amounts of work that need to be done, depending on the quality that you are trying to get. I think that we’ve achieved a very good quality with this movie. We’ve used ATVC conversion tool, that we’ve partnered with them over the last almost two years to improve, to get the quality levels that we have. Basically what we have to do is go through the complete movie, understand how we want the 3D to look. So basically, create a depth script for the movie, which was shot in 2D. There are a tremendous number of scenes like this one, where clearly he’s jumping forward, towards the viewer. So we can emphasize that in the 3d conversion. What I’m going to walk through this morning is how we actually do that. So after we have the depth script, and we identify that we want to robot to be coming forward, if you look behind him, there’s a view into a room that’s behind through the broken window. So what we need to do to be able to create 3D, is we have to identify the elements that we want to be brought out into 3D. For that we create what we have as “rough roto.” From that we’re going to create an individual rotomat of the image. So provide this as a guideline. In this case you see two: you see the pink of the robot and you see the blue of the window. Those are instructions to the roto artist, who is then going to create this image for the robot and this image for the window.
AP: You’re creating two different layers?
IH: Yes. Two full layers. Or in this case, three. So again these are the images that you use to create the depth.
AP: You have to do this frame-by-frame, right?
IH: Yes. So the point I was going to bring out now is, this is one frame of this shot. For every frame as he jumps we need the mats made for that for those object. What we do then is we separate those objects for the right eye and left eye. Then we have to fill in the miss date. Because obviously now I’ve created two images, there’s missing data that wasn’t captured in the 2D version. That has to be created. In our case, that’s very much created by the conversion tool itself. So it’s automated. Then we are able to get through the conversions quickly and provide a great deal of automation to the process. Currently that’s done by hand, by someone painting the pixels.
AP: It seems like the difference between good 3D and not so good 3D when you’re doing conversion. . .there must be a sort of art to this? People have to make decisions still? To make sure the effect is convincing and that it gets the effect you really want for maximum impact? So even though you use some automated tool, I image there must be some person still involved in the process?
IH: Right. Because the automated tools don’t always make the right decisions. The first step in the process is to make this depth script, second step is to create this roto so we get the mats that we need, the third step I’m going to show you now. This image here, if I flip back and forth, you should see, this is the left eye image – the original left eye – and this is the right eye created image. If you look, you can see that the robot is actually shifting. The way to look, if you see this vertical line, in the previous version, the left eye, he’s much closer to it. In the right eye he’s further from it. You have a shift in the robot, which is going to provide the depth. In this case, what we’re point out is that the image created as some artifacts – around his knee here and around his arm. Unless we clean those up – this is the cleaned up version, this is not – I think you can clearly see that the vertical line is much cleaner than it is in the original. So that’s an error made by the tool. That is then manually corrected. That is typically known as “paint.” So the tool does a great job of filling in lots of the areas, but in cases such as vertical lines, you do see areas that you have to fix. If you don’t fix those, you end up with bad 3D. We fix them, so we end up with a result that I think is very pleasing to the eye. The less of those distortions or artifacts that you have in the final product, is the quality level that you achieve.
AP: Good 3D as we would think of it?
AP: It’s really very interesting. Another thing I always wonder about, is how you make the decisions about 3D effects about something that looks natural or, I remember some of the older 3D films from the 1950s where they had these really exaggerated 3D effects. I imagine there’s some balance you have to apply there as well to make sure it’s natural, but not too much in your face. Or do you actually like the exaggerated effects from time to time?
IH: Well, I think we try to have a natural view to our (Fox) movies. If you look at the movies we’ve released. I mean, Avatar being the prime example of it, where you’re sort of looking into a world. We don’t have too many things jumping out at you. There are definitely shots where you want to emphasize that and give it a special effect. But mostly we’re looking at a natural effect, so that when you look at this in 3D, I think you’ll see the characters look very natural. They don’t look like cardboard cutouts, which you see in many not so good conversions. We’ve done, I think, a very good job of providing a very natural 3D view for the consumer. How we pick the movies, obviously, something like i,Robot, is a very popular movie with a fanbase. I think it was probably before its time and could have been 3D. There are lots of images like this, of major 3D events throughout the movie that we’ve been able to take advantage of to provide the fan and new fan a 3D experience that sort of emphasizes that.
AP: So it sort of changes the whole feel of the film, the whole experience of the film. If you watch it, especially if you’re familiar with it and seen it a dozen times, now you’ve got a chance to see it in a whole new way if you want.
IH: Exactly. You get to see it in 3D, you get to see all those shots where you have to imagine the 3D; you actually now get to experience the 3D. And for new viewers, it’s a great movie to enjoy to begin with and now with the 3D effects, I think it can bring it to a whole new audience as well.
AP: That’s great. One of the things we’re alway eager to see more of is we see more and more 3D titles coming to market. I think one of the reasons why this whole idea of conversion is being looked at so closely, is because if we’re dependent only on new releases to grow the catalogue, that’s going to be a slow pace. The idea of taking this enormous catalogue of content that you have available to you and then carefully selecting the films that you think are the right ones for the process, that allows us to build up our library a lot faster. That’s really the holy grail here, isn’t it?
IH: Correct. I mean, 3D really needs content, as well as devices. We have displays, we have Blu-ray players that can provide the real high value content, which is a great experience for people. We need to get more content and so this is Fox’s way of enabling that. To take great titles from our library and being able to present them to the new user of 3D.
AP: I can’t wait to see it in 3D. I haven’t seen it since if first was out, so it must be a completely different experience. Are there any other aspects of this when you talk about good 3D versus bad 3D? I know you pointed out the artifacts. Are there any other decisions you have to make? You mentioned the depth script, for example. It seems like you need a lot of know how for that process.
IH: It’s a trade-off for trying to provide a great experience for the consumers versus an artistic view of it. But also, the capabilities of the technology; as the technology improves, you’ll be able to do more and more and more. We’ve been able to provide a very good experience with the technology today; we’re still improving it. As additional titles are converted, I think you’ll see more improvement to the process. There is one other step I’d like to talk about. It’s a little difficult to view in 2D. This is a great shot: a shotgun blast, you have pieces of glass blowing all over the place, you have sparks. The conversion does a fairly good job of that, gives you a nice experience. But this is a key scene. It’s a shotgun blast. It’s maybe one of those where you do expect a little bit more coming out at you. We actually went in and took elements that were already in the scene. So if you look at the sparks, you look at the glass that’s flying; take those elements and we enhance them a little bit by adding some CGI, so they naturally fit within the scene. This is the original, this is the added CGI and when you look at the final, you can composite both and now you have elements where you can take the CGI and bring it further out in 3D. See these sparks here? Bring those out a little further, so when the shotgun blast happens, you have the conversion we’ve done, but we also have the ability to add to that conversion.
AP: So is this what’s involved when you talk about the depth script, seeing these keys scenes and making those decisions about things you really want to bring out?
IH: Exactly. Here’s all the things we want to bring out from a conversion standpoint, here’s some additional things that we want to do.
AP: That’s amazing. It’s really come a long way hasn’t it?
IH: It has. One of the things that we do, for Blu-ray in particular, there are many ways to convert, we choose to convert from a left eye, create a right eye. The reason we do that is Blu-ray allows you to play back 2D from one of the eyes that you select. So in our release of i,Robot, you can actually put the disc into a 2D player and still experience it in 2D. So we are using Blu-ray for the high quality experience, the high bit rates that you provide through the physical media. You could provide this through a streamed environment, but the bit rates are not going to match the quality levels that you can achieve with Blu-ray. Blu-ray is still a very effect means for us to distribute this.
AP: That’s one of the key points we’ve ben trying to make over and over again, is that there are things that packaged media can do, because of the highest bit rate you have available to you. When you think about a streaming network, you’re actually working in the opposite direction; the lower they can get the bit rate, the happier they are because you can get more content down the pipe. Whereas, with Blu-ray you have that one single channel in effect, to focus on and you’ve got a huge amount of bits compared to what the typical household can deliver in a streaming environment. That allows you to deliver the quality you’re after. It’s hard to imagine doing that in a streaming system at this point in time. Is that a fair statement?
IH: I think that’s a fair statement. I think there’s one other point, I mean, is obviously there are two images and you have to provide those two images. Blu-ray provides fullscreen 1080 per eye, and in a streaming environment you don’t have that decoder available, so you actually have to stream side-by-side. So, yes, not only do you have lower bit rates, you actually have lower resolution because you have half resolution. So the left eye and right eye contained within one frame, so you don’t have full 1920 by 1080, you have half resolution. That’s also why we prefer to provide this from a Blu-ray standpoint, because we have very, very high bit rates, we have full frame 1080 available to the consumer.
AP: It looks like a very convincing title. I’m really excited to see it. It just came out yesterday, i,Robot in 3D on Blu-ray 3D. I guess that wraps up out video portion, unless you have anything else to add?
IH: Go buy it and take a look at it for yourself.