NormalMap - 2D or 3D

Its been a while since I have been here, what with one thing and another...

My question(s) are based around normalmap's, and from the perspective of generating these for use within GS. To my understanding there are two types of Normalmap, which I call 2D and 3D although im sure this is not an accurate description.

Let me clarify a little. When you process a texture with a tool like Nvidia's Normal map plugin for adobe, you produce a 2D normal map. When you texture bake a normal map from a 3D object you create a true 3D normal map. It's usually easy to see the difference of these two types of map, 2D's are pink and blue and 3D maps tend to be yellow, green, pink, blue and everything in between.

For a better description please check out the tutorials I have posted on my clan site if you would like more information:

2D Normal Map
3D Normal Map

For the last few years I have been working with a AAA game engine developing expansions and other modifications, working very closely with the developers. Now during some testing with their engine I encountered a number of problems with normal maps and I wondered if anyone had some experience of these issues with GS and really just to explore the topic a little further.

So I noticed that with a 2D normal map, you will get display inacuracies, namely with the normal mapping having the opposite effect on the reverse of a model. i.e if the front of the model is raised, it's rear using the same normal map seems to be inverted. Now maybe I am applying this incorrectly and hopefully someone here will put me right. However I suspect I am correct in this matter as these 2D normal maps are just that, they are missing the 3rd axis and this is why the results are inconsistant and at best, poor looking.

So the obvious next step is to produce a 3D normal map, in my case, baked through 3D Studio Max's texture baking system, producing far better results. For example the human model I am baking at the moment is embedding much of the 18,000 polygon model into the 2000 poly one.

However in this other engine, i noticed some issues with 3D normal maps, when being rotated or moved, their shadows seemed to stay with the object, instead of re-adjusting. In other words a rotating barrel's self shadow rotated with it. Now im 90% confident this is simply bcz that particular engine's shader system was, flawed.

So I would be interested to know how GS handles these 3D normal maps.

I have taken the liberty of including a simple normal map in both 2D and 3D versions to wrap around a simple cylinder:


2D Normal Map


3D Normal Map

I have also added a small vid clip of these normals in action in the 'other' engine for reference. However in this clip the objects are not also rotating, so the shadowing is accurate here. Additionally I have added a link to another video of a fully normal mapped room, that is 1 model with a 1024x1024 diffuse and normal map.


Applied 2D & 3D Normal Maps
3D Normal Mapped Room Model

In short I was not happy with the way the other engine handled the new shaders and so I'm considering returning to GS. Your feedback would be very useful. Thanks in advance.

The first map above is not just a 2d normal map. It looks like a tangent space normal map and the second looks like an object space normal map. It really depends which one your shader supports. You can also create tangent space normal maps with 3d programs. The baking plugin from ATI (which is included with a wrapper module in Lightwave or Blender) allows to switch between tangent and object space normals.

So I would check that first.

Edit: Don't blame the engine for the shader or the wrong made normal map. That is just a very small part of the engine at all. It is just eye candy stuff that can be easily fixed. And the video with the room looks fine.

Yes, it's all about tangent vs. world space.. world space normal maps are on;ly useful if the object will never move or be oriented differently from how the normal map was rendered. Essentially you will never want to use world space normal maps except in very rare cases...additionally, you may need to change how the shader works to use them properly.

Tangent space normal maps are much more flexible, and all games that i know of that use normal maps use mainly tangent space maps.

All normal maps encode 3d data, so that's not a good way to look at it.

So using world based normal mapping for rooms, or for other non-dynamic objects but to use tangent based maps for other objects. But what about this problem of the normal map being convex on the reverse side of an object?

Applying the tangent map to a cube for example results (for me at least) with the map being inverted on opposite sides, and completly missing on other faces, when viewed in GS, or the 'other' engine for that matter. Is this a limitation of normal mapping?

As for my reasons for looking at switching back to GS as an engine, rest assured the problems with shaders are only a small part of the reasons. Originally I was approaching this for adding detail to static objects, it was only when I began moving them I noticed the 3D issue. Could the normal map shader not be addapted to cycle through the colours for orientation? I mean if Blue is east as the object rotates the hue does too, so tech you could offset the colour hue based on rotation...although this may be complete rubbish now i read what i have written

I havent had this trouble you talk about.. there are many things that can be wrong with a shader... sometimes there is an unavoidable seam if your model has mirrored UVs --though there isnt a good way to fix this for now...usually this isnt such a problem.

I dont ever reccomend using world space normal maps...bascailly you just need to experiment more and find better shaders

Well the results gained with world shaders on fixed objects, is considerably better than with tangent. Depth of detail can really bring the normal mapping to life. In fact I have seen little difference between the tangent normal mapping and standard bumpmapping... of course this may be bcz they are indeed the same thing

If you apply a tangent normal map to an object, illuminate one side and then take a look at the normalmap on the opposite side to the light. It is the opposite effect. I produced a simple object with a tangent based map and rotated it in GS. Close inspection i see the normal map on the reverse remains reversed and incorect as the object rotates.

Maybe you could point me at something that demonstrates this not to be the case... possibly the shader code I was using in my earlier GS experiments was incorrect or flawed itself, I was using one found somewhere on this site I believe...

I think that world based normals have their place, even if they are unsuited to dynamic objects, as the detail is considerably better and more realistic than any results I have had with a tangent based map.

Did you ever think whether the shader supports tangent space at all? When you use an object space shader with a tangent space map it could look very strange and false. I did that once with a shader from ello.

Well I have decided to do some tests, maybe you could point me at a good normal map shader?

I am baking a torso and head for testing and will upload screens as I go.

Ok so after a little discussion here, I delved back into the fray. Realising that I had generated my tangent maps using a photoshop plugin and not through Studio Max in the way that I was creating the world maps...

Clearly my frustration with tangent maps, (2D normal maps ), was rooted in the way I was creating my maps and not the way the shader engine handles the map. So a little lesson in, 'why not to try and teach your grandma to suck eggs..', later and I am once again excited about the power of normal mapping.

Thanks for your help guys I have taken the liberty of rating you guys highly for your help, for what thats worth.

For a limited time I have uploaded a demonstration of these findings and the comparison between world and tangent normal mapping... although it seems as usual I am the last to learn anything here... ...it might prove useful to someone...


Tangent / World Normal Mapping

From the look of it, I can say your normal map has inverted Y values. This is THE MOST ANNOYING THING EVER with normal mapping.. !!!!

ok, sorry, this is one of my pet peeves..I will explain:

there are two 'standards' for normal maps, the ATI standard with 'inverted Y' and the Nvidia standrard non-inverted Y. In practice, this has nothing to do with how hardware works or Nvida vs ATI, but simply how you write your shader. The upshot of this is, it is a completely trivial thing that causes all sorts of confusion. Basically, most of the shaders you will find on this forum that use normal mapping use the NON-inverted Y method for normal maps. You can usually specify this in whatever program you use to make the normal map.

The shader code I was using in this example is this one:

Code:

---

//  Basic Dot3 Bumpmapping.  
//
//  Calulates diffuse lighting from Sun and Level Light  
//
//  Needs:  A6.31 Comm+   DX9c  vertexshader.1.1 
//
//  Todo:  Lighting improvements
//
//  Eric Hendrickson-Lambert (Steempipe)
//
//  3/28/05:  Made changes for DX9 A6

---

So it may be a little out of date...

I tried another shader previously to this older one,
Code:

---

/**************************************************************/

// HLSL NormalMapping Shader

// -------------------------

// By:       Daniel "Rhuarc" Niezgocki

// Requires: []VS 1.1

//           []PS 1.4

// Skin1:    Texture

// Skin2:    Normal Map

//

// Usage:    Free to use, credit is nice, but not required.

/**************************************************************/

---

..but despite my Radeon Pro 9800 being PS 2.0 (I think thats correct ) I got a completely black model on my screen. Maybe you could help me out here, and point me in the right direction? Would be much appreciated.

Now that I have clarified the method it would be nice to put it into practise

i remember reading somewhere that Nvidia and ATI processors use normal maps differently, with one of the axises being inverted if u were to use a Nvidia-tested normal map on an ATI machine, and vice versa. i could be wrong.
and even if i AM right, it still shouldnt make ur model black. sorry, i can't help you.

julius

EDIT: whoops! matt already pointed out the differences.

My experience when getting a black model, is that the shader does not work with the card, but is not...bad code, simply that it doesn't run on that card.

I say this as running shaders that work on my Radeon, on a Nvidia Mx400 recently produced the same results. I suspect that the bump code I used that is producing black is doing something that my card doesn't like, even if its a PS.2.0