Books on Photorealism in 3D and CGI

If you’ve been following my blog and would like to know more about creating photorealistic 3D CGI renders, you can go straight to the source with these books from Amazon:


Crafting 3D Photorealism: Lighting Workflows In 3ds Max, Mental Ray and V-Ray

Light for Visual Artists: Understanding & Using Light in Art & Design

Color and Light: A Guide for the Realist Painter

Digital Lighting and Rendering

Photorealism: You Can Do It

Digital Texturing and Painting

Elemental Magic, Volume II: The Technique of Special Effects Animation: 2 (Animation Masters Title)

The HDRI Handbook 2.0: High Dynamic Range Imaging for Photographers and CG Artists

Physically Based Rendering: From Theory To Implementation

What does it take to make a 3D / CGI Object look Photorealistic?

Through my own experiments I have observed that the understanding of light is fundamental to achieving photorealism and it seems that many other 3D practitioners share this opinion.

Within the 3D environment, three aspects of light need to closely replicate their real-world counterparts:

  1. Light needs to illuminate the surface of a 3D object in a realistic way.
  2. The 3D object needs to prevent light from hitting other objects in the scene and create shadows.
  3. Light needs bounce off the 3D models in the form of reflections.

1. Light needs to illuminate the surface of a 3D object in a realistic way

HDRI

High Dynamic Range Images are a useful tool for creating photorealstic 3D renders as they create a natural light source and can also be used as an environment that appears within reflections. When using HDR Images, it’s useful to first light a scene by applying textures that have a solid colour of 50% grey to the 3D models. This allows the observer to adjust the exposure of the surrounding environment, replicating the light that was present when the HDRI was captured, before creating other textures.

Warm / Cold Lighting

When creating artificial lights, one should observe the colour temperature of those lights. Quite often, a warm (orange/yellow) light is used as a key light, and a cold (blue) light is used as a fill light. This is because our eyes are familiar with seeing the sun cast shadows whilst the blue sky casts soft light over the shadowed areas.

Colour Temperature

A 3D artist in search of photorealism should create lights that have the same colour temperature as their real world counterparts. The following is a small selection of real world lights and their colour temperature.

  • Candle flame: 1900°Kelvin
  • 100‐watt household bulb: 2865°Kelvin
  • Daylight: 5600°Kelvin

If a 3D artist in pursuit of photorealism was creating a directional light that was intended to emulate a sun, then the virtual light’s colour temperate should match that of its real world counterpart, i.e. approximately 5,600°Kelvin (depending on the time of day etc).

Volumetric Light

Any visible light within a 3D scene is created using volumetric lighting. A common use of volumetric lighting is to replicate a key light penetrating a dusty environment (More on dusty environments later).

IES Lighting

Even better than trying to match the correct colour temperature, an architectural 3D artist in search of photorealism should almost certainly make use of IES (Illuminating Engineering Society of North America) lights wherever possible. The IES have created a standard that allows manufacturers to record characteristics for the lights they make, such as colour temperature, falloff and visible light etc. These measurements are saved in a text file and made publicly available. 3D software (such as Cinema 4D) can then use these files to replicate a real world light exactly.

2. The 3D object needs to prevent light from hitting other objects in the scene and create shadows

Define Spatial Relationships

“When asking the audience to accept a scene that would otherwise strain its credibility, convincing shadow interaction can add an important piece of reality to help sell the illusion. If a production is supposed to be completely photorealistic, a single element such as a missing shadow could be all it takes to make your work look ‘wrong’ to the audience. Shadows serve the interest of adding realism and believability, even if there is no other reason for them in the composition” (Birn, )

Hard/Soft Shadows

When creating shadows, it is important to think about the source of light creating those shadows. A large light source that encompasses the entire scene would create a soft and even shadow, whereas a small distant source of light would create hard shadows. In nature, the sun casts hard shadows whereas the sky casts soft shadows.

Shadow Cookies

A cookie is used in the cinema to cast a shadow with a predefined shape. For example, if you wanted an actor to look like he was in a forest, you might cut the shapes of tree branches out of cardboard and place them between the key light and the actor. This would cast shadows that look like tress into the scene.

In 3D, if an artist is trying to composite a 3D object into a live scene, such as a photograph or video, shadow cookies cast over both the 3D model and the original scene can help to behind the two media together making it difficult for the audience to distinguish between them.

3. Light needs bounce off the 3D models in the form of reflections

Surface Texture

Although this project hasn’t gone into great depth in regard to surface textures, they are however extremely important when trying to achieve photorealism. All objects, except perhaps a black hole, have some amount of reflection, however, all have differing reflection properties. For example, a chrome lamp will have a very hard reflection, whereas a wooden picture frame will have a much softer reflection.

Fresnel

In addition to how hard or soft a reflection is, the amount of reflection on most objects will change depending on the angle you look at it. This is achieved in the 3D environment with the use of a Fresnel layer applied to the texture.

4. Other things to Consider

Colour

When creating textures for 3D models that will appear within a HDRI environment, it is helpful to use colours that match the hue and saturation of the HDR environment. Once a 3D image has been rendered, Hue and Saturation adjustments applied to the entire image help to blend the two media together.

Camera Artefacts

One problem with 3D renders is that everything produced is beautifully clean and sharp, as if it had been photographed with an extremely superior lens and sensor. In order to fool the human eye into believing something was captured with a camera, some of the unwanted by-products of cheaper lenses need to be replicated. This includes over/under exposure, chromatic aberration, noise and grain, vignetting and silvering. Stylistic choices such as depth of field should also be observed. And when shooting film or animation, other artefacts such as motion blur should also be added.

Edges

It has also been observed that in the 3D environment it is possible to create perfectly square edges which, if magnified an infinite number of times, would remain perfectly square. In the real world this is less often the case as edges tend to be worn and/or rounded. To make 3D objects appear real, hard/square edges should be avoided.

Randomness / Chaos

In addition to avoiding square edges, and perfectly clean/crisp renders, some of the random chaos of the real world should also be introduced. For example, rather than using an algorithm too create a brick wall that is perfectly straight and where every brick is exactly the same shape and size, there should be some variation. In addition to this, dirt should be added into a scene.

Dirt

When creating dirt with a photorealistc effect, you should paint dirt onto a model by hand. Burns (p229) correctly states that you should“choose dirt maps that add specific, motivated detail to your objects. Think through the story behind all of the stains and imperfections on a surface – something has to cause any dirt, scratches, or stains that you would see”. 

Dirt should only be present on the surface of models, but should also be present floating in the environment in the form of dust, steam, or similar. Volumetric lighting is a good way to achieve this.

Perception

The final thing to consider is human perception.

Scale

It’s possible, although I haven’t as yet been able to prove or disprove this theory, that scale plays an important role in creating an illusion of photorealism. We know that if we see a 60 foot gorilla on the screen, it is most likely computer generated rather than a real photograph.

Unreal

It appears that it is easier to fool the human brain into believing something is real if the brain has fewer points of reference. Take for example a human hand, creating a 3D hand that an audience believes is real is extremely difficult as it is something we spend a great deal of time looking at and accordingly we have extensive points of reference. If, on the other hand, I was to create 3D model and said it was a newly discovered creature that was found deep in the ocean, it would be easier to fool the mind into believing it was real as the brain has fewer points of reference.

That said though, it is still important to look to the real world for influence and reference when creating something that is fictional.

Improving on this Research

It’s difficult to find a way to improve upon this research, as has already been said, it appears that many other practitioners already share my view and any experiments that I have conducted myself are simply reinventing the wheel. Is it possible that all of the problems have already been overcome?

At present, I don’t feel I have explored the subject deeply enough to be able to offer any new insight that hasn’t already been discussed elsewhere.

What I propose to do now is produced some 3D renders that illustrate all of the points that I have made above. These renders will then be presented for assessment as a ‘body of work’. It is hoped that whilst creating some new renders, problems might arrise that haven’t already been tackled. However, I expect that this is more likely to happen if I approach a novel situation that other practitioners haven’t yet tried to create in 3D. This could be fun 😀

To try and streamline this process, I find that although I can build 3D models, it takes me a great deal of time. In light of this, I might try and create some scenes with simple geometry, such as a piece of jewellery, a planet, or perhaps to take a scene that I have created previously and try to make it more photorealistic.

Summary of Findings to Date

Before commencing with any further experimentation, today has been put aside for summarising my findings thus far.

In my initial experiment with the grey spheres I observed that an understanding of light is fundamental to achieving photorealism. Within the 3D environment, three aspects of light need to closely replicate their real-world counterparts:

  • Light needs to illuminate the surface of a 3D object in a realistic way.
  • The 3D object needs to prevent light from hitting other objects in the scene and create shadows.
  • Light needs bounce off the 3D models in the form of reflections.

High Dynamic Range Images are a useful tool for creating realistic lighting as they create a natural light source and can also be used as an environment that appears within reflections. When using HDR Images, it’s useful to first light a scene by applying textures that have a solid colour of 50% grey to the 3D models. This allows the observer to adjust the exposure of the surrounding environment, replicating the light that was present when the HDRI was captured, before creating other textures.

When creating textures for 3D models is it important to use Fresnel Reflections as this type of reflection more closely replicates how light bounces in the real world. To understand what Fresnel reflections are, imagine looking at a body of water on a Sunny day, if you looked across the surface of the water you would see a lot of reflection and little of what is beneath the surface. If, on the other hand, you looked down at the water from above, you would be able to see what was below the surface and less of the reflections. This property of light needs to be emulated in the 3D environment.

In addition to this, discounting black holes, everything in the real world has a reflection, although some have very little. With this in mind, everything in the 3D environment should have some amount of reflection.

When creating textures for 3D models that will appear within a HDRI environment, it is also helpful to use colours that match the hue and saturation of the HDR environment. Once a 3D image has been rendered, Hue and Saturation adjustments applied to the entire image help to blend the two media together.

One problem with 3D renders is that everything produced is beautifully clean and sharp, as if it had been photographed with an extremely superior lens and sensor. In order to fool the human eye into believing something was captured with a camera, some of the unwanted by-products of cheaper lenses need to be replicated. This includes chromatic aberration, noise and grain, vignetting and silvering. When shooting film or animation, other artefacts such as motion blur should also be added.

It has also been observed that in the 3D environment it is possible to create perfectly square edges which, if magnified an infinite number of times, would remain perfectly square. In the real world this is less often the case as edges tend to be worn and/or rounded. To make 3D objects appear real, hard/square edges should be avoided.

It’s possible, although I haven’t as yet been able to prove or disprove this theory, that scale plays an important role in creating the illusion. We know that if we see a 60 foot gorilla on the screen, it is most likely computer generated rather than a real photograph. Similarly, in this animation of a spider, the illusion of realism is ruined because we know that spiders are not this big.

Spider Animation

Finally, it appears that it is easier to fool the human brain into believing something is real if the brain has fewer points of reference. Take for example a human hand, creating a 3D hand that an audience believes is real is extremely difficult as it is something we spend a great deal of time looking at and accordingly we have extensive points of reference. If, on the other hand, I was to create 3D model and said it was a newly discovered creature that was found deep in the ocean, it would be easier to fool the mind into believing it was real as the brain has fewer points of reference.

In account of all this, what’s next?

Although I’d very much like to continue developing the antagonist, as can be seen in the project plan, module deadlines dictate that I begin researching the findings of other practitioners and comparing their results to my own.

Before this is done, I will first address my objectives for satisfying the learning outcomes to ensure that they are all being achieved.

Is Fresnel Reflectivity the Best?

In my previous post I had noted that fresnel reflectivity was producing the most photorealstic results, however, before dissmissing other reflection types, I decided to conduct a further experiment.

In the renders below, the first image (top-left) shows three spheres rendered without any texture applied to them. It was interesting to note here that the spheres are very dark. When trying to achieve photorealism in this image previously I had noted that the spheres seem quite dark. Without any texture, it makes it far easier to see how dark they are. In the future this will prove useful when making adjustments to achieve correct exposure from the surrounding equirectangular panoramic HDRI sky dome.

In the second image (top-centre), adding some colour to the speheres supports the illusion of photorealism. This is perhaps because white is more reflective than black and some of the colours of the surrounding environment is seen to be reflected in the spheres. For example, there is a green tint in the bottom of the spheres where the surface of the filing cabinet is being reflected onto them.

In the third image (top-right), the spheres are 100% reflective. Contrary to my previous observations, this image does not appear as photorealistic as the white spheres. I believe this partly to be due to the reflection being too clean; in reality there might be smudges of grease, dirt and the like, you’d also be able to see a camera or similar in the reflection. In addition to this, the reflection in the coloured spheres is almost like a fresnel reflection which could account to its being more photorealistic.

However, this does confirm that fresnel reflectivity is superior, however, fresnel reflectivity alone is still flawed, as can be seen in the following image (bottom-left). In the image titled 100% Fresnel, the centre of the spheres has no reflection whatsoever. This is not a realistic effect.

It seems then, that it is a combination of all of these things that will have the greatest effect when trying to achieve photorealism in 3D renders, as can be seen in the final render.

Final Render of the Polished Balls

Final Render

Comparing Reflections

In an earlier post I predicted that reflective objects would appear more photorealistic than non-reflective objects. Having conducted a short experiment, I can confirm that this is true.

Looking at the three renders below, it is the cube without any reflection that appears to have the least photorealism, the cube that is completely reflective is closer to photorealism, and the final cube, with fresnel reflectivity, is an improvement on them both.

This shows that reflection does have a vital part to play in achieving photorealism, but further experiments are required to identify if fresnel reflectivity is the key.

Completed Photorealistic Render

and a brief summary of observations

Earlier today I made a comparison of renders and identified that the centre of the spheres needed to be more reflective, the reflections were too blurry for such a polished surface, and the balls were still a little too dark. With all of these issues corrected, here is the final render.

Final Render of the Polished Balls

Final Render

I must admit, I’m quite happy with the image and feel that I have been able to create something that is photorealistic. My only major concern is the light from the window being reflecting from the surface of the green filing cabinet. In reality, the balls would have hidden some of the surface from this direct illumination and it would have been far greener. Focus could now be given to correcting this problem, but my feeling is that the time required wouldn’t justify what little would be learnt in the process. For now, having realised that this happens is sufficient as it enables me to take this into consideration when creating a background photographs in the future.

From this initial experiment I have observed that, in this image at least, understanding light is fundamental to achieving photorealism. It is a combination of light bouncing off the surface of the 3D object as reflections, light illuminating the 3D object, and the 3D object preventing light from hitting other surfaces.

What’s next then? At this point one obvious option would be to spend some time researching other practitioner’s observations of light. Before that however, I would very much like to spend some more time observing fresnel reflectivity; the experiments to date have been based on a single, insular situation and need to be developed further to accommodate different shapes with different surface properties. It would make sense to conduct a survey to see if people believe the renders to be computer generated or photographs of real world objects, however, my feeling is that more could be learnt from this if a collection of renders was first prepared.

First of all, now that a scene has been created with some success, I’d like to put some different 3D models into the scene to see what happens.

Render Comparisons

With some progress made it’s time to compare the renders and try to identify what qualities of reality have been achieved and what is still missing.

The first image shows above (top left) is the original photograph that is acting as a point of reference for trying to emulate reality. Rather than concentrating on making an identical copy of the photograph, the intention was instead to emulate the elements of the photograph that make it look real.

The second image (top middle) is the background image that I am using in my 3D renders. A decision was made to use a background in this way in order to give the experiment a more narrow focus. As my understanding of what is happening improves, I will later attempt to create an entire scene from scratch.

The initial test render (top right) was the initial result of the experiment, seen for the first time, without any adjustments.

Interestingly, when showed this image, I was asked why I had taken a photograph of three balls. Haha, already I had fooled somebody! Admittedly though, this person had been shown the image on the small screen of my mobile phone, but this did prove that something in this image makes it begin to appear ‘real’. it doesn’t take a genius to work out that it is the reflections in the balls. This coincides with my initial theories relating to achieving photorealism by studying light.

The biggest failing in this image is the lack of shadow below the balls which is causing them to look like they are floating. Again, a product of light, or lack thereof.

Some time was spent producing an improved render (bottom left). The shadows below the balls definitely help to sell the image. Indeed, whilst sat in my office at work my colleagues found it difficult to believe that this is a computer generated image. Yet still, it lacks something, there is still something about it that doesn’t seem quite right. Firstly, the green filing cabinet seems to be very desaturated compared to the rest of the environment and the balls seem to be in a darker environment than they are supposed to be. In addition, after giving some thought to fresnel reflectivity, I realised that the reflections are still wrong.

In the further improved render (bottom middle)  the shadows seem more natural now that the artificial software lights created in the 3D environment have been turned off, and whilst there is some odd striping at the front of the cabinet, it has at least got its’ colour back.What’s lacking in this image is that whilst the fresnel reflectivity around the edge of the spheres is better, the centre of the spheres need to be more reflective. The reflections are also far too blurry for such a polished surface and in comparison to the original photograph, they still seem a bit too dark. Time for one more render I think…