If you’ve been following my blog and would like to know more about creating photorealistic 3D CGI renders, you can go straight to the source with these books from Amazon:
I’d hoped to gain more insight from reading Jeremy Burns chapter on Decals and Dirt, but found that I had already been using what he refers to as Decals in my projects. In the example robot image below, I had already applied decals (stickers) to my robot, however, what I hadn’t done was added dirt.
In this example I’ve probably gone a little over the top with the dirt but it helps to illustrate the point I’m making.
When I showed the animated version of this robot to a fellow artist, he immediately noticed the lack of movement in the robot’s hands and how clean the robot looked in comparison to the background. Although the film noise/grain and motion blur that had been added were helping to blend the two media together, it seemed the texture on the surface of the robot’s skin, was too clean and shiny to begin with.
In the example above, I’ve used Ambient Occlusion (a tool for finding edges between two surfaces and is often used to emphasize shadows) to create the dirt which in my opinion is a inaccurate way to achieve the effect. When creating dirt with a photorealistc effect, you should paint dirt onto a model by hand. Burns (p229) correctly states that you should “choose dirt maps that add specific, motivated detail to your objects. Think through the story behind all of the stains and imperfections on a surface – something has to cause any dirt, scratches, or stains that you would see”.
In the example above, dust would have fallen and settled onto the robot from above, this would mean that the robot’s shoulders, head, bridge of it’s nose and chest would probably have accumulated the most dust. Similarly, water dripping from the ceiling would have created vertical streaks running down the robot’s body.
Painting in these extra details will require considerable time but it’s this attention to details that will eventually sell the illusion of photorealism.
In my previous post I demonstrated that Replicating Digital Imaging Artifacts helped to achieve photorealism.
To further this research I have been experimenting with replicating artifacts found in digital film such as grain, and motion blur and adding them to 3D animations.
The examples below show a single frame from an animation, one with the added artifacts and one without.
When looking at a single frame, the motion blur effect seems out of place, but when watching the animation and the motion blur of the original camera movement becomes apparent, it is clear that the replicated blur helps to create the illusion of realism. In contrast, in the animation without the added motion blur, the sharpness of the cg elements stand out from the noisy, blurred background and begin to hinder the illusion of realism.
As well as motion blur, other artifacts have been added according to previous findings such as chromatic aberration, vignetting, and noise.
The affect of sound will be discussed in a future post.
Throughout this process of trying to emulate reality I have speculated that it is not the real world that should be emulated, but realism as found in a photograph. To this end I have attempted to replicate some of the artifacts that occur in digital photography in order to improve my previous best attempt at replicating reality.
The Image ‘Without Post-Processing‘ shows the previous attempt at achieving photorealism.
In the image ‘Camera Artefacts‘ I have added some chromatic aberration (The red fringe on the edge of the balls), I have scaled the image up and down, compressed it and decompressed it, and used run length encoding to degrade the quality of the image, and have also added some noise. All of these effects would occur naturally when working with digital photographs.
In addition to this I have also desaturated the image and adjusted the colour balance to make the highlights a little more blue. The benefits of this are that all of these changes affect the entire image, not just the 3D elements. As both the background and the 3D elements have been affected by the same processing, it becomes more difficult for the eye to distinguish between the two forms of media.
For reference, the original photograph has been included also.
Whilst there is not as much noise or chromatic aberration in the reference image, when comparing the two artificial images, it is the image with the added artifacts that is most convincing.
In my previous post I had noted that fresnel reflectivity was producing the most photorealstic results, however, before dissmissing other reflection types, I decided to conduct a further experiment.
In the renders below, the first image (top-left) shows three spheres rendered without any texture applied to them. It was interesting to note here that the spheres are very dark. When trying to achieve photorealism in this image previously I had noted that the spheres seem quite dark. Without any texture, it makes it far easier to see how dark they are. In the future this will prove useful when making adjustments to achieve correct exposure from the surrounding equirectangular panoramic HDRI sky dome.
In the second image (top-centre), adding some colour to the speheres supports the illusion of photorealism. This is perhaps because white is more reflective than black and some of the colours of the surrounding environment is seen to be reflected in the spheres. For example, there is a green tint in the bottom of the spheres where the surface of the filing cabinet is being reflected onto them.
In the third image (top-right), the spheres are 100% reflective. Contrary to my previous observations, this image does not appear as photorealistic as the white spheres. I believe this partly to be due to the reflection being too clean; in reality there might be smudges of grease, dirt and the like, you’d also be able to see a camera or similar in the reflection. In addition to this, the reflection in the coloured spheres is almost like a fresnel reflection which could account to its being more photorealistic.
However, this does confirm that fresnel reflectivity is superior, however, fresnel reflectivity alone is still flawed, as can be seen in the following image (bottom-left). In the image titled 100% Fresnel, the centre of the spheres has no reflection whatsoever. This is not a realistic effect.
It seems then, that it is a combination of all of these things that will have the greatest effect when trying to achieve photorealism in 3D renders, as can be seen in the final render.
In an earlier post I predicted that reflective objects would appear more photorealistic than non-reflective objects. Having conducted a short experiment, I can confirm that this is true.
Looking at the three renders below, it is the cube without any reflection that appears to have the least photorealism, the cube that is completely reflective is closer to photorealism, and the final cube, with fresnel reflectivity, is an improvement on them both.
This shows that reflection does have a vital part to play in achieving photorealism, but further experiments are required to identify if fresnel reflectivity is the key.
…and why they lack photorealism
In my previous post I was reasonably pleased with how close I had come to a photorealistic render and decided that the next task would be to render some different 3D models in the same scene as had been used previously.
In doing so I have been able to confirm some of the beliefs I held when planning how to replicate the qualities of reality in 3D CGI.
In this experiment my intention was not spend time adjusting each render in pursuit of photorealism but instead to make as the minimal changes required to obtain some kind of output. In most cases this required only scaling the models to fit on top of the filing cabinet. In account of this the results are quite pleasing. This is however, not to say they are without flaw; this short experiment has allowed me to observe some other qualities of reality that are not currently present in my renders.
Looking at the 3D renders above, it appears to me that, for a number of reasons, the man-made objects (the dice and tins of spray-paint) appear more photorealistic than the organic objects (the spider, frog and snails).
A possible reason for this is that the man-made objects are far more reflective than the organic ones, and as such, the light bouncing off their surface is very much in keeping with their surroundings. It would be interesting to compare a simple render where in one image the object is very much reflective, and the other has no reflection whatsoever. According to my findings, the reflective object would appear more real.
Scale is another reason I believe the organic objects to appear computer generated; it is obvious that the spider is computer generated because it is far too big. In order to fool the brain into interpreting something as real, perhaps the scale of that ‘something’ needs to be in keeping with its real world counterpart.
It was also noticed that colour seems to play a vital role in achieving photorealism. When comparing the frog to the snails, despite the snails being cartoon-like in nature, it is they that appear more photo-realistic.
When writing about my creative journey, I was pleased to see that Weta Digital (the animation studio that created the cave-troll in The Lord of the Rings) used the colours of the surrounding environment when creating the texture for the cave-troll’s skin. The reason I found this pleasing was because I had adopted the same approach when trying to achieve photorealism in an earlier experiment where I had animated and composited a CG frog. In this experiment I utilised colours from the surrounding environment when creating the texture for the frog’s skin. When comparing the renders above, I find the cartoon snails fit into their environment better than the frog because the saturation (vibrancy) of the colours more closely match the surrounding environment. Further experimentation is required to confirm if hue and saturation aid photorealism.
Another reason I feel the snails look more real is because they are not real. Confused? Me too…
Let me try and explain; if the snails are cartoon-like in nature, they must therefore be be man-made. If they are man-made, then they don’t need to look real because in reality, there is no such thing, and therefore, our brains have fewer points of reference for comparison. Consider this; I’d bet that if I modelled a realistic snail, exactly the same as one you’d find in your garden, it would be more difficult for me to fool you into believing it was real.
It’s also possible, although this requires far more research on my part, that one of the problems with the organic objects is that no light is passing through them.
When you put your hand in front of a strong light, you can see the light pass through your hand, and as a result of the light passing through the blood in your hand, the skin turns red. It could be that the organic objects in my renders might look more real if some light was allowed to pass through them. In 3D and CGI terms, this is known as sub-surface-scattering.
The final problem that I have observed in the renders is that the 3D models are too clean and perfect. In the background photograph there are artefacts that look like they might have been created by the camera or are a direct result of resizing the background image. There is some noise and grain in the picture, some areas of the background are overexposed, and, some of the colours are bleeding into each other, whereas in the 3D objects, everything is sharp and crisp. To get a photorealistic render, my feeling is that some of these artefacts will have to be artificially created.
I know that in my previous post I said that after producing these renders I would give fresnel reflectivity more focus, however, I think the next logical step from here would be to try and prove or disprove some of the observations made above.