The visuals I had used to present Woodbound with so far were too noisy for what I want to accomplish, so recently I sat down and rewrote the way I'm tackling lighting and tonemapping. Here's a before & after (before>after):
I posted an in-progress gif, and got a pretty mind blowing response, prompting me to bump up this website on my priority list so that I could have a good platform for the much requested write-up on my scene. Which, incidentally, is what the post is also for.
Going (back) into editing the lighting style, I wanted to draw inspiration from a look similar to older animated productions. The best example that I can think of is in the old Scooby Doo cartoons from the 70's, where a villain would be hiding in a scene, and the idea was to reveal the character by having their eyes move or have them peek around a corner, but the suspense of having it happen at the end of a scene was kinda ruined by the fact that the static parts of the scene were painted (beautifully) while the pieces that were animated were line drawn. (also beautifully but with less detail for practicality.) This was of course a limitation of the time, it would have been impractical to spend hours on every frame making the level of detail match with the unchanging background. And my highlighting this isn't an attempt to knock the quality of the productions, if anything it highlights the look that defined those cartoons.
The way to transfer this production earmark to modern 3D that made sense the most to me was to follow the same principles when keeping in line with lighting theory; any objects that are static and motionless on a frame-frame scale could have slightly more elaborate lighting and shading than something that moves around on a frame-frame basis. This is visualized in the gif above with the rocks and boulders having a more detailed level of shading than the grass and trees.
I still wanted to keep a fairly cel-shade inspired look overall, so even the more elaborately shaded rocks lack fine details typically derived from things like tessellation, parallax occlusion mapping or even normal maps, which are absent from the scene in it's entirety. To maintain the flat lighting and also drive the tonemapping, I utilized Thura Oo's Semi CelShade PostProcess Material, with these settings in an instance of that master material:
I did not bake any lighting in my scene, as I've taken to using dynamic lighting for the majority of my time developing in UE4. I utilized a Directional light for the sun, with an intensity of 5, as well as a skylight using a blank cubemap in lieu of a scene capture (easier to iterate on lighting without having to recapture the scene, especially because I'm not setting up any reflection behavior) with an intensity of 0.4. My post process settings include disabled auto exposure, bloom intensity of 2.0 and threshold of -1.0, and AO power of 4.0.
Another seemingly minor piece that I think plays a huge part in the scene is the visible specularity lines passing over the grass as a body when the wind blows. This is achieved by mapping and panning a modulated voronoi noise texture along global coordinates, and using the black and white values of the R channel as an alpha in a lerp to mask in the lighter colors, simulating that sheen that passes over, as seen in the material below:
As you can see, the diffuse information is solely based on solid colors, and the normals are simplified by disabling tangent space normals and plugging a 0,0,1 Constant3Vector node into the normal slot of the material output panel. This removes excess noise that would normally be generated between each blade of grass, allowing the field of grass to be shaded as a body rather than as tons of individual pieces, keeping in line with the idea that a by hand animator would likely light the grass as a group for practicality as opposed to shading each blade as it moves frame by frame. (Shadow casting is also disabled in the grass mesh.) The grass is then placed using a landscape grass layer node in the landscape material, keeping the grass diffuse in the landscape material the same color as the grass primary color, so that the meshes are tougher to distinguish from the ground they're on, and making the whole body of grass look thicker than it actually is, keeping a wild look while saving on performance.
The trees are nothing special geometrically, just a basic trunk mesh with some branches, and some planes with a branch texture, warped and arranged on the limbs and set up with spherical normals. The material is slightly more texture based than the grass, but there is a fare amount of base color overriding happening, by way of a lerp node. I highly recommend the Two Sided Foliage shading model.
The boulders are largely procedurally modeled. (outside of UE4.) I use Blender, but the same principles I used could be applied in Max or Maya. I took a cube, subdivided it 5 times, applied displacement using a voronoi noise texture mapped along global coordinates (I feel like I've typed this before...) and then decimated it down. As you scale up your procedural instances, you'll need to create separate noise texture instances with larger mapping scale values so that the dents from the voronoi don't get too jagged/noisey.
That was the entire procedure for 3/4 of the sets of rocks I made, but with the last set, (the tall, pillar-esque pieces) I added the additional step of carving off some geometry from the sides of the rock to give them some nice slated features.
The rock material within UE4 is, again, pretty basic. In fact, it's the most basic surface material in this scene because it aligns the most closely to the basic lighting principles in mind for UE4's realistic lighting system:
The idea behind the fresnel lerp setup is to multiply the brightness of the rock color through the diffuse to simulate an "edge lighting" popularly seen in dramatic cel shaders, but it's all simulated through basic math within the diffuse output. This also makes it super cheap to render.
Although I'm very humbled and grateful for the response I received for that post, I don't believe anything here is particularly novel or complicated. I'm happy to share my methods in the hopes that it can help others to achieve the stylized look they're after. Please feel free to post a comment or email me at firstname.lastname@example.org if you have any questions. If you found this tutorial helpful, please do me a favor and go like my game's Facebook page, as I'm looking to make some significant strides in the coming months, and community support is hugely encouraging.