Canyon Scene Reconstruction: Using Reconstructed Meshes for Shadows and Reflections

Photo of Rick Barrett

Instructor Rick Barrett

Share this video
  • Duration: 08:43
  • Views: 1981
  • Made with Release: 19
  • Works with Release: 19 and greater

Apply Compositing Tag options to Reconstructed Meshes in order to create Shadow Catchers and Reflection Sources.

In this video, you’ll learn how to re-project the video footage onto a mesh generated using Cinema 4D’s Scene Reconstruction. You’ll also learn how to set up C4D’s Compositing Tag to treat the mesh as a shadow catcher, allowing 3D objects to cast shadows on the landscape without actually rendering the reconstructed mesh. The reconstructed mesh can also be reflected in CG objects, so they appear to be a part of the video environment.

show less

Transcript

In this video, we'll look at one of the most basic things that you can do with your reconstructed mesh. Re-projecting the footage back onto the mesh and then allowing it to act as a shadow-catcher and a reflection caster. But first, let's go ahead and clean up some of these examples so that we just have the point cloud and the mesh itself. And, in fact, I'm going to go ahead and hide the point cloud for now so we're just working with the mesh. We'll go ahead and turn off the solo mode as well. We'll project our video footage back onto the mesh in a couple of steps, and the first step is to actually bring it back into the background. You're seeing the footage now because it's being projected via the motion tracker. But if we turn off the Show Image option here in the motion tracker you'll see that our footage no longer exists in the scene. In order to bring it in, we can create a background object simply by clicking this button here in the footage tab. That's going to create a material for the footage and a background object and apply that material to the background object. Now, any time we apply a material to a background object it is applied in frontal projection. This basically applies the texture from the current viewpoint. We know that the current viewpoint matches the footage itself because we're looking through the solved camera. So, we can CTRL+ drag this texture tag up onto the scene mesh and we'll project the footage on to the mesh as well. The problem with this technique is that as soon as you look through a different camera the projection breaks down because the projection is always happening from the standpoint of the current view. And I'm going to hide the background objects so this is a little bit more obvious. So you can see that the canyon footage stay stationary even though I'm rotating the canyon around. We can solve this by using a different projection type called Camera Projection. This is basically the same as frontal projection, but it allows you to specify the view from which the texture is going to be projected. So I'm going to jump down here into the texture tag and change it from Frontal projection into Camera Mapping. The great thing about this technique is that we already have a solved 3D camera that we created as part of the reconstruction process. So we can just drag the solved camera here down into the texture tag and set that as the source of the projection. We also need to click the Calculate button in order to adjust the film aspect for the aspect ratio of the footage itself. And now you can see that the canyon texture is being locked to the appropriate portions of the mesh and we can actually rotate and get a little bit of camera movement here without the texture actually tearing apart. Now, if you rotate too far you'll start to see smearing. And that's the nature of camera projection and there's methods to overcome this smearing. But, typically, what you're going to do with camera projection is just adjust the camera position or rotation slightly and you can totally get away with that. Now, even if I'm not going to adjust the camera position I like using camera mapping here because it allows me to move around the canyon in editor-view and the texture remains locked onto the canyon itself. But for now, we'll jump back into the Solved Camera. The next thing that we want to do is set up this canyon to receive shadows so that we can add 3D objects into the scene and integrate them completely into the reconstructed footage. And for that, we're going to first need to create an object that can cast a shadow. So I'm just going to create a large sphere. We'll drag the sphere up here and make it bigger. And I want to go ahead and create a light source to cast the shadow. And, in this case, I'm just going to use an infinite light which represents a light like the sun that's infinite distance away. So the position of the light source itself doesn't matter. We can simply rotate the light in order to choose the direction of the shadows. So I'm just going to rotate this around so the shadows are kind of casting from this way down on to this wall of the canyon. And we need to go into the light settings and turn the shadows on. So we'll switch this shadow into Area shadows. Now, if we go ahead and render what you'll see is that we are getting the shadow on to the canyon and the shadow is actually deforming with the shape of the canyon, which is totally what we're looking for. The problem is that we're seeing the flaws in the reconstructed mesh and also the canyon itself is being very heavily shaded by the light source and that's not what I'm going for here. So, first, let's go ahead and bring the background back in. And what we want to do is set this mesh so that it receives shadows and casts reflections without actually being visible in the scene. In order to do that, you add a Compositing Tag in Cinema 4D. So right click on the scene mesh object and go into the Cinema 4D tags and choose the Compositing Tag. The key option here is Compositing Background. This basically removes the object itself from view but keeps it present for the purposes of compositing. If we render now you'll see that over here things have completely cleaned up. The mess that we're seeing over here is because the canyon is casting shadows on itself. And we can solve that simply by unchecking the cast-shadows option. Now the canyon won't shadow itself or if you want to you could have the canyon cast shadows but not have it be self-shadowing. And now you can see that that totally cleans up all of the mess of the canyon. And we've got the sphere totally integrated into the canyon casting a shadow on the canyon wall. Now, of course, you're going to want to go into the light source itself and adjust the color of the light, the angle of the light, the density of the shadows, in order to really better integrate this scene and make it look like those spheres are part of the scene. In fact, we can get the reflections from the canyon as well. Let's go and create a new material and we'll go in and activate just the reflectance channel. I'm going to remove the default specular here and simply make this a shiny GGX, but we'll increase the roughness a bit. I'm going to apply that on to the spear and now you'll see that we're getting a reflection of the canyon itself. The problem is that everything above the canyon currently is black and to solve that we're going to need to add a sky object into the scene. And the way I like to do that is by adding a new sky object and applying an HDR Map. HDR maps are basically specially designed textures that give you a look of an environment. And there's quite a few that are available in the content browser. You can see I'd already searched here for HDR and got this whole list of HDR maps. And I'm just going to use this one of a cloudy day. And so I'll create a New Material. Double click to open that material. And here in the color channel, I'm just going to drag this cloudy VHDR over into the texture slot. And you'll see the hand turn into an up-pointed finger. And, there we go. Now, we need to apply that texture onto the sky itself but now we're not going to see our background anymore because we're seeing the sky instead. So to solve that what I'm going to do is add another compositing tag here on the Sky. And we'll just turn off the Scene By Camera option. Now when we render you can see that we've got a pretty nice reflection here as well as a nice shadow cast onto the wall of the canyon. This sphere figure actually looks like it belongs in the video footage, and that's what we're going for here. Now, in the next tutorial, I will take this a step further and create a bunch of spheres that will fall down and collide with the canyon walls.
Resume Auto-Scroll?