NAB 2016 Rewind - Joshua Michie: Designing for VR with Cinema 4D

Photo of Cineversity

Instructor Cineversity

Share this video
  • Duration: 38:17
  • Views: 12382
  • Made with Release: 17
  • Works with Release: 17 and greater

Joshua Michie of Apiary discusses the creation of a GearVR experience “Antum” using Cinema 4D and Unity.

Joshua Michie of Apiary discusses the creation of a GearVR experience “Antum” using Cinema 4D and Unity. Joshua explains many of the limitations and challenges of VR development, and the plugins and workflow he used to develop a VR experience. Joshua used Cinema 4D’s MoGraph tools to create the cave-like environment for Antum, and
NitroMan’s Magic Merge to combine the MoGraph elements into polygon objects. Cineversity’s CV-SmartExport makes it easy to export selected FBX files from Cinema 4D directly into the Unity project. Once in Unity, Joshua shows the basic process for setting up the VR camera, materials and shaders.

02:21Antum Project Overview
07:53GearVR Limitations
09:37Modeling the Environment
13:58Mesh Checking
14:37UV Mapping
19:14Getting C4D Geometry into Unity (CV-SmartExport)
25:21Detail Work
32:09Unity Materials and Shaders
36:22Antum VR Preview

Recorded Live from NAB 2016 in Las Vegas.



Thanks for coming out guys. My name is Joshua Michie, and I'm going to talk today about C4D and this wonderful new medium, VR. Oh, prograph guys. You guys helped me out a lot with this. So I'm going to show you the pipeline, all the way from the creative concept; the team that we assembled. And then as we were getting into the development of what would be, I guess, a game. VR is like a game. The pipeline that we learned, some of the face plants, and learning lessons so that when you get into it, you're not wasting a month on forum hunting, and/or maybe not as much time as I was. Before I get into it, for those online, thanks for coming by. Take a screen grab. These are the team that helped me out. It's good to have a couple 3D artists, and absolutely integral to have an interaction designer, and a developer that probably knows C#. I'm going to be bringing this into Unity. You don't necessarily need a Unity developer, but having a C# guy, you can't make anything without that magical code. So without further ado, I will drop my wonderful little reel on you, and we'll go from there. ♪[music]♪ Thanks guys. So Apiary is a new thing that's happening with me and said team. We do a lot of this commercial work, and need something where we can build our own IP, and start telling stories on our terms. And this VR thing was perfect, because there's an open market. I'll be talking about Samsung Gear VR. In that store, there's a lot of opportunity for people, for motion designers, for 3D artists to get out of the broadcast pipeline, and transition into that game VR, and play around and experiment. It's an open game right now. So that's a little bit about us, and our first project is Antum. I'll show you a video, a screen grab video at the end of the presentation. This is loosely based around Plato's allegory of the cave, and it's one of three parts that we are making. Each section, or each part is exploring one, either headset or a different medium. As the team gets more and more comfortable with VR, so too do the complexities, and interactions, and stories, and it started with a simple sketch. Nothing crazy. In Toronto, which is where I'm from, we have a really strong motion meetup, Toronto motion meetup, and we have a really strong VR Toronto meetup as well. A lot of people jump between the two. In Toronto, we do have a lot of people interested, and there's only a couple studios that I know of, that are actively making. So it's inspiring to have just an open game and get into it. So me and Nixson, who was, I guess, my partner in crime with this one. We developed simple sketches. We found some stuff. We wanted a big monolithic open cavernie old space. Knowing that with the limitations of VR, you can't have vast vistas. You have to be thinking about intimate spaces. I love basalt, which is that where that waterfall picture is there. Also because I knew I could mograph the shit out of that. And then on the right hand side, which we're still working on, I want to get light projection. It's a cool graphic design and light projection that people can interact with as they're exploring this space. This was the first concept sketch that popped out in the night after a few drinks, and getting into that art board. From there, Nixson and I just further developed it, thinking like, how we can get big, and was this enough that I could rope in Jeff Briant, who's an amazing motion designer, and get him excited. And Jeff Moberg, who is the sound designer on the team? As soon as they saw this little frame, which really wasn't...nothing really to talk about. But it was enough that it gave everyone a vague idea about what we wanted to do, because none of us had done this before. So we were like, "We want to do something like this. We'll maybe make a map. We'll go from there." Then Jeff came with these awesome style frames in Octane. We were really excited like, "Oh, there is going to be like light halos, and depth of field, and displacement maps." All of these things, you can't do in Unity right now, especially with the Gear VR. It's limiting right now. But also, I don't know why you'd want depth of field when you put on the headset. Unless we can track your eyes, there's no reason to have that. But this got us in some kind of dark, nice direction that we wanted to go down. Then Nixson being the interaction designer, rather than doing a storyboard, because this is an open-ended exploratory narrative, we had to build a UX map. Which has changed a little bit from this, but this was enough for us to approach Unity and start putting in assets, and different modules, and prefabs. And looking for scripts that might be able to help us out with this. We get to the wonderful headset. It is one of the more technically limiting of the headsets, but we chose that deliberately because if I had little sticks from the HTC vibe, or even the ability to start moving around and exploring a space with Oculus, I knew the scope was going to start creeping up very big, and we only gave ourselves two, three months to get this thing out the door. I also have the phone, so that helped out as well. I didn't have to spend $6,000 on a headset. So when you get into it, rather than going into the step by steps, screen grab this thing, take a picture of it, especially the Google creating a Gear VR App in Unity Free by eVRrdayVR. Just follow that step by step and you should get your SDKs, and the code and everything talking to each other, which took an awful long time, because I did not follow instructions to the letter. He's got a bunch of other stuff that I suggest checking out, and SideloadVR App, which is a magical little thing on your phone, which gets the number that your phone needs to go talk to Oculus and upload your APKs. I'm also going to be talking about the C4D...oh sorry, Cineversity CV Smart Export, which was a huge time saver. At least when we were setting up our sketch scene, and Magic Merge, which was a God send. So things to think about before you get into it, and we were already way into it when we came about these things. Low poly, high texture. That doesn't necessarily mean low poly style. We wanted to stay away from that, and if you build your prefabs, or your models small enough, you could build your textures at 1048, or 2048 by 2048. But you're going to have to compress it down in Unity, because the phone can't handle it. You have to keep to that 100k polycount, and 100k vertice, which I don't know how to give you an idea about what that looks like. The scene that I have can give you a vague idea, but until you are in it and you're making it, I don't think you're going to understand what that means. Because you'll quickly go over that budget the first time around making your world. And prefabs, which are like instances, but for Unity. So I'll get into that as well, and if it doesn't move after you put your stuff in there, make it static. So your batch calls, which is the game or the graphics card, and the CPU pinging to see how many materials are on something, how many different models. So keeping models the same, and reusing the same textures keeps the memory running high, and prevents you from dropping down to 60 frames a second. When that happens, everyone gets the vomit comet almost immediately, and forget OBJs, just FBX is the industry standard. There's this weird goblin where it wants to make a parallel universe of whatever asset you've made, and mirrors it. I don't know why that happens, but it happened enough that just stop it. Don't do that. So we had a rough idea where we were going in building the scene, and here's the sketch that we roughly built. You can see it's pretty messy. This is roughly a little's up to that like 100k polycount and vertice. That means what the camera sees in Unity is what they call a vert, so what's being rendered. I'm going to talk a little bit about how we built this stuff in here. This was pretty simple when we started out. Pretty simple with the layout. Just took a poly pen tool and started sculpting out the rough like scene. Then once we knew what that was, I started putting in these single objects, and I'm going to do that right now for you. Back to the basics. Nothing too crazy. Start with a shape. Bring it down to six, so you've got your little dude there. Open up your Cloner, drop it in. Nothing new. You can do a small count there, and something like that. So that's a 30, so they...oops. I've got something like that, and mush them together, add a random effector, and only set for the Y, because I want them up and down. I'm building that basalt column stuff. So something like that, and then I was happy with that. Duplicate it, pull it back, upset, offset a little bit, and I'm making sure that the geometry intersects with each other, because I'm going to be bringing in Magic Merge, which is going to bring this stuff all together, and make it one object. Multiple objects in Unity, it would render every single one of these things, and you would kill your scene really quickly. So I've got a nice video run through of... it took about an hour to get this thing built up, and into the UV mapping. So you've got the stuff together, select it all, alt G or option G. Bring it into a null. Make everything an object. Grab all those guys, put them into...oops. Put them into one null, Shift C, bring in Magic Merge if you've got it in there. It turns it all into one object, which was great because now it's just a hollowed out thing, and you can get pretty complicated with your sculpting in your materials. When you do this, it just gets rid of all the interior gobbledygook. For example, just get rid of that guy. You can see now that there's some stuff, some intersecting probably because I didn't do the best of jobs. You can see here, for example, there's a little bit of a gap. So you got to be careful when setting up your geometry, because then it just won't merge it all together. It thinks it's a negative space. So I've got this really rough geometry here, and we've got to bring it into the UV because this thing is destroyed as a UV map. Putting a shader on it, it's not going to look very nice. Because Jeff Briant was very nice in being the guy that wanted to paint all these things. I needed to at least set up a UV map for him, so that he could go play, and make something really cool, which I'll show in a bit. This part, I don't really care what program you're in. UV mapping is not a fun, it's an ordeal. I do have a little bit of a video to show you that fun time. But before I do, I pressed shift M , and then I opened up the mess checking. And this is great for finding out if you've got rogue polygons, or edge points which you don't really need, or non-manifolds. These are all terrible little objects that goblins that show up. If you get things too close, so I don't know. Sometimes if this area right here was just a little bit too close, and you Magic Merge it, you might end up with either bad polygons, or something that might be a non-manifold, because Magic Merge isn't's a little confused. There's a little bit of a discrepancy there. So mesh checking is great, and before you go into your wizard, I don't need that. That guy right there. Either set up 1024 as your UV map, or 2048 so that at least you've got a bigger map to play with, and you can always downgrade that and compress it in Unity. Then we've got some stuff. This one's not too bad, but still pretty disconnected, and you can play around with...oops. Make this cubic, or box, or flat. It's really just it depends on your shape you're trying to find a way of getting as much real estate as possible, and rather than show you how slow I am at that. I've got a video so that you can watch the joys, and pain of UV mapping. ♪ [music]♪ So in this case, Magic Merge did a really good job, or I made a point of making sure there was no too many in between floating polygons. Cleaned it up. This made it really, really light. ♪[music]♪ This is where everything starts going sideways. So what I'm thought process with UV mapping is like grab things that are the same. Grab them, put them off to the side, and just keep doing that as like a puzzle piece, and then try and resize and bring things back together. All the while trying to figure out like, "Is this box projection? Is frontal projection? Is this going to help me out, or is going to make it worse?" I'm also on Skype talking to Jeff as he's laughing at me when I'm getting this thing set up. Because I'm like, "Well, if I get all the top hexagons into one area, you know where those are. Right?" He's like, "Yeah, I guess. We'll see how that works when I come back in and paint it." ♪[music]♪ At this point, I still haven't seen that I've smushed some of the geometry. Those hexagons are supposed to be perfect square, cubic. They are smushed like an iris. So that's going to deform the mesh quite a bit. But apparently I'm really hopped up on trying to get these long columns together. 1024 is a pretty small map, so I'm really trying to squeeze as much geometry, as much real estate as possible, trying to get rid of as much negative space. ♪[music]♪ Then finally I was like, "Oh, there we go. Okay." ♪[music]♪ Then just gave a rough material for him as a suggestion so that when he went in the painting, he had a rough idea. That's the nightmare of UV mapping. So once you've got that done. I think we've got Octane. Jeff came back and made this wonderful...this is not the same object. This is something else we worked on, but it was the same process. I think we made five iterations of these different clusters. Let's see if I can bring up Octane and check it out. Maybe, yeah. He came back with this, and I think he used Substance Painter, because they have PBR shading and that was something that in our long researching in Unity, it likes PBR shading, and with a new 5.4 that's about to come out. They've come leaps and bounds in the graphic improvements. So he made this guy, and we were pretty happy. So back into the scene. Part of the research we had was, either bring this scene into Unity as a Cinema 4D file, except we were having troubles, because there's just too much geometry here. It took way too long, or it would just not show anything when you brought it into Unity. That's where we had this Magic CV Smart Plugin, which is a Cineversity CV Smart Plugin. Where are you? There we go, CV Smart Export, my bad. Entire scene into one file, and in this case, make sure that you've got everything in your knolls, and you've got them named in a way that you understand, because it's going to grab all the materials that would be inside of your knoll. Then make separate folders for Unity. It saves hours of time at least in the initial set up. However, because of prefabbing, you're taking not just this guy as himself, or by himself. You're taking the whole cluster. If you want to iterate like we did in trying to figure out what things look like, it's better just to have one of these guys exported in, one of these guys exported in, and then individually move and build your scene in Unity. So plugins, CV Smart and that's not mine. Let's go to... I've got a folder here. In this case, you have to create a model. The first time you open up or build your Unity scene, you've got to tell it where to go with the models. I'm going to delete all this stuff, or I'll do it again. I'll try cave three or two. There we go, six save and it's done everything. Now I'm going to open up Unity. The first time you set up Unity, it's best to pull one of the 3D cameras directly from the prefab. Taking a little bit of time. There we go. Or you can import it from your assets, import new asset or package and it will say, "Characters." You're going to open that guy up in there. Okay, that's all right. I've already done this. In your characters, you'll have your prefabbed first person, prefabbed first person controller. You can drag that in there, get rid of that camera. Don't need that. In my models, everything is imported here, so KV2, I can bring that in there, and now I've got all of the scene more or less built here. You can already tell there's a lot of geometry. That's because Unity takes your quads, and turns them into tris. So how we went about cleaning this up. It was mostly just that these objects were pretty good. They didn't have too many polys or tris, and oh yeah. Here we go. Let me set that camera up. There we go. Make a light. Just sort of something to see. There we go, that's better. There we go, and you can see here it's a little bit small, but you can see that I'm already over my... I've got 107 tris, and 153k verts. So the easiest way for this is there's a plugin in Unity called Simplygon, and you can bring this online into their cloud, and they'll reduce or create LODs, which are, in this case, we didn't really need it. We need the low SLOD. In games, there's different tiers of geometry the farther away from a camera you can get. So the closer you are to the camera, the more geometry you'll get, the farther away, it can be mushy and not so nice, because you're not really going to see it. We use that tool to clean this up quite a bit, and keep it below 100ks. Oops. So you see here, this is what I was talking about with that prefab. These guys here come in a full wall. So this whole thing has been prefabbed or instanced. Rather what I would have is just one of these guys. So what I would do is once we had...once you're in the VR scene, and you're looking around, or you're looking at this. You've got a gauge of scale, and do you want this thing there or that thing there? Start deleting everything. Make one of these objects rather than a bunch, and then go back into C4D, and use Magic Merge again so that you can remove any back facing geometry, because... I don't know if I can show you the overdraw. Where was that? Right here. You can see the overdraw that's happening here. Unity renders from the farthest to the closest, and when you do an export like this, it doesn't necessarily know what hierarchy it needs to go into. You're going to have to script this out a little bit, but I can already tell back here, I've got a whole wall section of geometry that you're never going to go see in the game. So we went back in, chopped that out, cleaned that up a little bit, and same thing with those giant columns. We ended up chopping the whole backside, because there's no reason to see that stuff. There's another thing I want to show in C4D, was, sometimes you might have...and this was supposed to be like a landslide, where all these old basalt structures would fall, and break down. It's really close to the player, so we wanted something with nice detail. So the same idea again. Put that guy in there, and it's just a sheet. Nothing crazy again just playing around, sculpting with some basic geometry. Then I did the same thing again with the mograph. It was an ongoing thing. Doing that all over again . I'll just put that in there. Oops, make that cloner an object, and then drag your landslide in there, and you've got these guys here. This was before we realized we needed less than 100k. This was definitely our geometry killer, but in some cases, this might be a quick way for you to build your own asset, so I'm going to go through and do that. Put a random effector on that guy as well. In this case rather than position, I want rotation, and in the cloner, I want the surface. Just keep bumping that count up quite a bit. Play with that rotation a little bit. Make something random. I don't know, something like that. Scale it up just a tiny bit. I'm going to do something like maybe 100, 200. So you put a lot of geometry in this. The same thing happened again. We were happy with this. Grab these two things. Put the landslide into the Cloner, make sure everything else is an object as well, which is a lot now. C4D hates objects. It loves high polycounts, but it hates objects. So this is a really nice beefy machine, but very quickly with my little laptop, this was slowing down to a crawl. Originally what I wanted to do is just use Magic Merge, but now I'm going to create a lot of unnecessary geometry any time that this thing has an intersection. So I needed to figure out how to get that landslide to shrink down. There's two ways of doing this. We ended up using a script, because for some reason, I did not see the magic button on it. So you've got all your polygons selected, and then you can extrude, right click, extrude and push that down. This is what we were seeing. It was like there's a big cavern, and a hole in here. So we needed to close that up. We could have gone and done that, creates caps. However for some reason in the midst of building this, we did not do that. I was working with Gene on this one, Gene Magtoto. So he made a script for me. So I'm going to show you how to do that. Script folder, run scripts, oops. Console, that's what I was not looking for. There we go, script manager. He built a Python script, and I've got that written down here, because the internet is awesome fast here. It's called Dupe Poly Place in Code. You can find it on his website. He's got a bunch of awesome plugins. I think it's Then select everything here...oh sorry, not that. That will break everything. There we go. Control C, Control V. I did not copy that. There we go, and then execute that, or click and drag that little icon so that I don't ever have to open up that script manager again. So in this case, I'm going to...well, I did that backwards. Back it up, there we go. So you've got it selected. You just press dupe, and now I've got this original geometry, and then I can click and pull where I want it. I've got a little bit of control in deciding, and I've got one whole object. Anything that's inside this thing now will disappear when I do my Magic Merge. But you need to be careful of this. Oh man, totally like. In gones or normals, sorry. The normals are backwards here, so in Unity it just makes them invisible and you can't see them, which took us a little bit of time to realize like, "Why can't we see our normals, and why aren't they facing the right way?" So in this case, what I would do is select these guys. The ones that I want, because I know these are the odd men out. Cool. Not that guy, and then normals, reverse, select all and then do it again. Now they're all the same. That's good. Everything is in your Cloner, and then you can Magic Merge that, oops. Shift C, Magic Merge, Enter. It might take a little bit of time. Oh, it's not too bad. Mapping, and know I've got this wonderful little crazy geometry. So in this case, we would have had to increase the geometry on the landscape. We ended up not using this at all, because the tri, polygon count was insane, and as I'm willing to bet, if I bring this into that UV edit mode, and do that mesh check again, I'm not going to see very nice things. Now I'm starting to see bright polygons. I can select those, and if I start deleting those, I'm going to start getting holes, lots of edge points. So this is bad geometry, and I want to show you because this is what you're not supposed to do, if you're going to get too crazy with your asset building. I think that is Unity. I think that's about all I really needed to talk about. Oh, I might talk about materials a little bit. Unity and C4D have similar materials, but they're different. So I had shown earlier that Octane, this guy right here, this basalt cluster. You can't really import your Octane materials. Sometimes normals bump maps. They do this weird smoky thing. It just doesn't look very good, and if your geometry is too big. Like we had painted the cave base like the big outline, the walls, and used a 1024, 2048 UV paint. It just looks really mushy and crappy. So in this case, for big objects like this, it's good to go find tileable shaders. So I'll just briefly go into this, not too much. We're now getting more and more into Unity, and that's a whole other conversation. In this case, I just wanted to show you here's how you would go about building a scene for C4D. So in my assets, I know what I'd like to do right here. A material and a shader, which are two different things. Normally...there we go. Click on your material, and it will have its different kind of... I guess these would be like, you want something that's subsurface or you want something that's reflective, or sometimes you might have like some weird programming projection map idea, and you can script shaders to go do that stuff. Whereas the material is what you're going to attach on to your geometry. I guess I'm going to import that guy in there. The other thing is if you're going to CV Smart Export, in this case, I've only got one object. I wouldn't want to do that, because it's also going to bring in all this other stuff. It's going to bring in your cameras, it's going to bring in your Octane sky. We don't need any of that stuff. So in this case, I just want to export. FBX, and I'll put that right here. Where'd it go? Oops. That's the wrong side there. In the assets models, and I'll put it right here for now, and call it that. There we go. Bring everything in. Find out what that guy is kicking. He doesn't have any material or anything attached on to him, so he still brought it in. So I guess it's best just to get rid of anything if you're going to be exporting into Unity that you don't want, because it's going to be there even if it's hidden. So it's got a shader that's been premade. Unity gives everything shaders, and you've got these different areas, these different maps that you want to attach, either your normals or your heights, or your elevdo [sp], which would be your RGB. But the normal map in VR is kind of a hit or miss. I guess it's just the nature. You don't need to fake height, you don't necessarily need to fake those textures, because often in Unity when you're putting it on, or in Gear VR, when you're putting it on, the spatial awareness... it's almost a distraction, and it's a serious CPU hog. So that's roughly what we went about doing as we slowly built the scene. I will bring in the magical cave, and before I get into this. Screen capturing for VR is not fun right now. So what I did here was just rebuilt the scene that we had for Android, and then brought it into the PC build, and then I just grabbed Camtasia and did a screen capture, which is why it's a little bit cloudy around the edges. But here is just a really rough run-through of where we're at with the current development. When I'm done here, you guys can check out the... I've got the headset, and you can go check it out what this thing looks like. ♪[music]♪ So each of these little sections have a different piece of this greater song that we've broken down, and Jeff's designed it in a way so that you can go look in one area, and do your experience, reset the scene, and then look at another area. And as the different sounds or sections meet up, you can change the tone of the song that you're listening to. Right now, we've just got random twinkly interactions. We are going to be getting into controlling that, but that's through scripting and that's in Unity. That alone is going to take a month; just to do the proper interactions that we want. That's that. We are hoping to release in June. If you want to see a little bit more, and maybe send us an email. If you're interested, I can probably send you an APK, if you give your number to your phone.
Resume Auto-Scroll?