Cubemap normalization

Posted By: Excessus

Cubemap normalization - 07/29/06 13:16

I'm trying to get my normalmapping shader with blinn-phong specular to work on ps 1.1. I have to normalize the halfangle vector in the pixel shader but because I can't afford another normalize() (max 10 instructions..), I think I'll have to use cubemap normalization.

What I have is this:
Code:

c-script:
bmap NormalizeCubeMap = "normalize.dds";
..
material abc
{
skin1 = NormalizeCubeMap;
..
}
..
bmap_to_cubemap(abc.skin1);

hlsl:
texture mtlSkin1;
samplerCUBE NormalizationCubeMapSampler = sampler_state
{
Texture = <mtlSkin1>;
MinFilter = Linear;
MagFilter = Linear;
MipFilter = Linear;
AddressU = Clamp;
AddressV = Clamp;
};

pixel shader:
InHalf = 2 * texCUBE(NormalizationCubeMapSampler,2 * InHalf - 1) - 1;


Explanation:
I first define a bitmap that contains the normalized vectors. The sides are in the following order: x, -x, y, -y, z, -z (I found this image on the web). Then I assign that bitmap as the first material skin. Then I use bmap_to_cubemap to convert the bitmap to a cubemap, I'm not sure this is needed so I also tried without that instruction and it doesn't work either. I then set up the texture and sampler in hlsl. In the pixelshader I try to normalize the vector InHalf. InHalf has been shifted to the range [0..1] so I have to shift it back to [-1..1] by 2 * InHalf - 1. Then I use that as the 3d coordinate to lookup in the cubemap. I then shift the resulting vector from [0..1] to [-1..1].

The results are wrong. It's not just inacurate it just gives incorrect results. I'm pretty sure the normalize.dds is correct (atleast it contains normalized vectors, I checked that) but I'm not too sure about the order of the sides.. Can anyone spot the mistake here?

btw, my shader works fine when I use normalize() and ps_2_0 as compile target.
Posted By: Matt_Aufderheide

Re: Cubemap normalization - 08/02/06 04:10

Can you show your shader code where you try to normalize using the lookup? I might be able to figure it out..

btw, why do you need ps 1_1?
Posted By: Excessus

Re: Cubemap normalization - 08/03/06 19:33

Hi Matt,

I use the blinn-phong algorithm that uses a halfway vector between the view and light directions.
Here is the code:
Code:

float4 BumpSpec_PS( in float2 InTexN : TEXCOORD0,
in float3 InHalfVec : TEXCOORD1,
in float InShadow : TEXCOORD2) : COLOR
{
float3 BumpNormal = 2 * tex2D(NormalMapSampler, InTexN.xy) - 1.0;

// Normalize halfway vector with cubemap:
// InHalfVec = 2 * texCUBE(NormalizationCubeMapSampler,2 * InHalfVec.xyz - 1) - 1; // doesn't work..

// Ordinary normalize:
InHalfVec = normalize(2 * InHalfVec - 1); // Works only in ps_2_0

// Calculate specular:
float Specular = pow(saturate(dot(BumpNormal, InHalfVec)), SpecPower);

// For debugging you can output the would-be normalized half vector:
// return float4(normalize(2 * InHalfVec - 1).xyz, 1); // How it should look
// return 2 * texCUBE(NormalizationCubeMapSampler, 2 * InHalfVec - 1) - 1; // How it looks with cubemap normalization.

return Specular.xxxx * InShadow * SpecularCoef;
}



Thanks for lookin into it. I could also post the whole shader, or even a demo implementation, if you can't find the error this way and you have the time to look at it..

I want my game to support ps_1_1 hardware and I'd like to enable normalmapping for the first LOD on those cards, so thats why I need ps_1_1 support.
Posted By: Matt_Aufderheide

Re: Cubemap normalization - 08/03/06 21:46

Quote:

// InHalfVec = 2 * texCUBE(NormalizationCubeMapSampler,2 * InHalfVec.xyz - 1) - 1; // doesn't work..




EDIT:
I'm not sure but I think you cant do this kind of texture read in Ps 1_1 .. why not try doing your half vector calculation (2*half-1) in the vertex shader, then just do the normalize and expansion in the pixel shader? In other words, this is a dependent read because you are changing the texture coords in the pixel shader before loading the texture. this can only be done via some specifc instructions in versions lower than 1.4 (i could be wrong, i dont really understand this very well)

Can you post some screens or something showing with and without cubemap normalization? Also post a jpg of your cubemap, it may be set up wrong for A6; i had that problem myself.

Are you mip-mapping your cubemap? that can cause visual errors..
Posted By: Excessus

Re: Cubemap normalization - 08/04/06 10:00

Infact the halfvector is being calculated in the vertex shader, but since texcoords are automatically clamped between 0 and 1 in ps_1_1, I had to compress the halfvector to [0..1] and expand it again in the pixelshader. It doesn't seem to be a problem though, the compiler doesn't complain.

This is what it looks like in ps_2_0 with normalize: Image
This is what it looks like when I use cubemap normalization: Image
And the cubemap: Image

I'm not mipmapping the cubemap (I don't use bmap_to_mipmap and it's not a .dds image).
Posted By: Matt_Aufderheide

Re: Cubemap normalization - 08/04/06 12:42

Ok your cubemap is correct.

Try this out just to test.. use the cubemap normalization but compile it for ps 2.0 and see if the same problem is there.. if so then we know for sure that something is wrong with the math..
Posted By: Excessus

Re: Cubemap normalization - 08/04/06 12:55

I get the same results when I compile cubemap normalization under 2_0. I have also tried not packing the vector (which is not possible when compiling in 1_1) and testing it with normalize() and cubemap normalization, same results so it's not the packing/unpacking of the vector either.
Posted By: Matt_Aufderheide

Re: Cubemap normalization - 08/04/06 13:54

well it seems like we have ruled out almost everything...

Try this, just try using your cubemap as a projected texture, just to test to see if its really being turned into a cubemap.

To do that, just calculate some coords in your vertex shader like this:
projectcoord = VertPosWorld - lightposition;

then just use texCubeProj() in the pixel shader..

Thn just set your final color to this color.. only for testing to rule out any problems with the cubemap itself.. (like maybe its backwards or not being turned into a cubemap for some reason..)
Posted By: Excessus

Re: Cubemap normalization - 08/04/06 14:47

Ok, I calculated a 3d texcoord in the vs:
OutCoord = PosWorld - float3(53, 103, 30);

Output it in the ps:
return texCUBEproj(NormalizationCubeMapSampler, float4(InCoord,1));

This is what I get when I use bmap_to_cubemap on the cubemap: Image

I assume that is needed, but just incase, here is what I get without bmap_to_cubemap: Image
Posted By: Alexander Esslinger

Re: Cubemap normalization - 08/06/06 16:54

Your normalization cube looks wrong to me.
Posted By: Excessus

Re: Cubemap normalization - 08/06/06 22:18

Oh, what is wrong with it then? Do you happen to have a working one?

I did some calculations and the vectors in the file do seem to be normalized. As I sayd I'm not sure about the order of sides.
Posted By: Matt_Aufderheide

Re: Cubemap normalization - 08/06/06 23:09

Alexander, I dont think its wrong.. it look like the same one i used before..
Posted By: Alexander Esslinger

Re: Cubemap normalization - 08/07/06 11:43

Like this: http://www.cescg.org/CESCG-2002/GSchroecker/img33.png

The order should be -x, +z, +x, -z, -y, +y - with the orientation shown in this picture.

To test your cubemap, just place a sky-box in your level and assign it - it's as easy as that to check if it alligns correctly.
Posted By: Machinery_Frank

Re: Cubemap normalization - 08/07/06 12:16

If it really works like a sky box then it must be right, behind, left, front, down and up. At least this is the order when I create sky cubes for Gamestudio.
© 2024 lite-C Forums