Vector Displacement, realtime Blendshapes

Posted By: jumpman

Vector Displacement, realtime Blendshapes - 08/13/18 04:57

Hi friends, I might be biting off more than I can chew, but I was wondering if this is possible.

With tex2dlod, and pixel shader 3, you are able to sample a texture within the vertex shader, and move vertices around based on reading textures. This is how they make oceans move, terrain deformation and etc. However, as a character artist, im really interested in seeing if I can implement blend shapes/morph targets. A big reason is to add facial expressions.

Right now, I can do very simple stuff with texture fetch within the vertex shader. I could do something like, move the vertex up X amount of units based on the red channel of the texture. Move this vertex down X amount of units based on the blue channel of the texture.

But as you can see, you cant store negative values in a texture, especially any formats that gamestudio can accept.

How can I offset a vertex, in the XYZ axis, but also displace them in the negative direction?

Ive looked up vector displacement formulas in google, and I dont know where to start.
Posted By: Superku

Re: Vector Displacement, realtime Blendshapes - 08/13/18 08:21

You can just store the data in the vertices itself, then you don't have to sample textures.

Code:
typedef struct {
	float x,y,z;	// position in DirectX coordinates
	float nx,ny,nz;	// normal
	float u1,v1;    // first coordinate set, for textures
	float u2,v2;    // second coordinate set, for shadow maps
	float x3,y3,z3,w3; // third coordinate set, for tangent vector and handedness
	float tu4,tv4;  // 4th coordinate set, for additional data
} D3DVERTEX;

That's the D3DVERTEX vertex declaration from atypes.h - it's wrong though, you cannot really access and use tu4,tv4 like that. A8 is using the following instead:

Code:
const D3DVERTEXELEMENT9 VertexA8[] = {
	{ 0,  0, D3DDECLTYPE_FLOAT3,  D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION,     0 },
	{ 0, 12, D3DDECLTYPE_FLOAT3,  D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_NORMAL,     0 },
	{ 0, 24, D3DDECLTYPE_FLOAT2,  D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD,     0 },
	{ 0, 32, D3DDECLTYPE_FLOAT2,  D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD,     1 },
	{ 0, 40, D3DDECLTYPE_FLOAT4,  D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD,     2 },
	{ 0, 56, D3DDECLTYPE_UBYTE4N, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_BLENDWEIGHT,  0 },
	{ 0, 60, D3DDECLTYPE_UBYTE4,  D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_BLENDINDICES, 0 },
	D3DDECL_END()
};

Might look confusing at first but what it tells you is how the last 8 byte ("float tu4,tv4") are actually used.
I'd recommend using ent_buffers over ent_getvertex to get D3DVERTEX pointers. You could for example use the 2nd UV set to store data.
Posted By: jumpman

Re: Vector Displacement, realtime Blendshapes - 08/13/18 18:04

wow thats really interesting Superku, thank you!

How would you force information into those floats? Would it depend on the modelling program you use? And wouldnt this information be discarded once you save it as an MDL file?

Also wouldnt this also mean the bytes could only store a single set of floats? Which means if we could force information in there, that it would only support one blendshape?
Posted By: Superku

Re: Vector Displacement, realtime Blendshapes - 08/14/18 04:51

You'd have to do that via code (for example ent_buffers, see previous post), yes.
That's true, the available data is very limited (for my cases it's enough most of the time).

To help with your original question: Just use 128 as your "0" in your texture channels, then everything below 128 is negative, above is positive. In the vertex shader:
float4 texData = tex2Dlod(...);
float3 displacement = (texData-0.5)*someScalingFactor;
Posted By: jumpman

Re: Vector Displacement, realtime Blendshapes - 08/14/18 21:46

Thanks for helping with that snippet Superku.

Im having trouble wrapping my head around how to not only generate the correct vector displacement map within mudbox, but also write the vertex displacement snippet in my vertex shader. This may be over my head. I might give it a few more shots.

Code:
float4 bloatShape1 = tex2Dlod(clothSampler6, float4(InTex.x,InTex.y,0,0) ); 
float3 bloatMove1 = (bloatShape1-0.5)*20;


I cant just InPos.xyz += bloatmove1 , this will offset everything in one direction.

I dont know the correct map to generate within mudbox (16bit), and I dont know how to offset a vertex position based on the RGB components. Im not finding formulas online either.

Do you think you have pseudo code to manipulate a vertex on RGB? What would be a good sample texture in mind that you think would work?
Posted By: Steempipe

Re: Vector Displacement, realtime Blendshapes - 09/09/18 07:17

Helpful info, SuperKu.
© 2024 lite-C Forums