thankyou very much for the idea of vertex normals, i probably would not have thought that on my own and kept trying different flags and mabe even some shaders and stuff before giving up!

Here is a video of the new look thanks to modifying normals:
http://www.youtube.com/watch?v=6RvUfXQ8_ZA

an example code for how I implemented one of the edges (works for terrains with 33x33 vertexs)
Code:
void avg_normals_N(int x, int y)//NORTH EDGE SMOOTH NORMALS (SHADOWS)
{
	int tx; //vector counter
	VECTOR nml[2];
	VECTOR mid_nml; //average height of touching vertex's
	CONTACT* c1; //contact vertex for Terrain
	
	
	if(y<TERRAIN_COUNT-1)
	{
		for(tx=1;tx<32;tx++)
		{
			vec_set(mid_nml,nullvector);
			vec_for_normal(nml[0],Terrain[x][y],1+tx);
			vec_for_normal(nml[1],Terrain[x][y+1],1057+tx);
			vec_add(mid_nml,nml[0]);
			vec_add(mid_nml,nml[1]);
			vec_normalize(mid_nml,1);
			
			c1 = ent_getvertex(Terrain[x][y],NULL,1+tx);
			//vec_set(c1.v.nx,mid_nml);
			c1.v.nx=mid_nml.x;
			c1.v.ny=mid_nml.z;
			c1.v.nz=mid_nml.y;
			ent_setvertex(Terrain[x][y],c1,1+tx);
			
			c1 = ent_getvertex(Terrain[x][y+1],NULL,1057+tx);
			//vec_set(c1.v.nx,mid_nml);
			c1.v.nx=mid_nml.x;
			c1.v.ny=mid_nml.z;
			c1.v.nz=mid_nml.y;
			ent_setvertex(Terrain[x][y+1],c1,1057+tx);
		}
	}
}



notice how I commented this line:
//vec_set(c1.v.nx,mid_nml);
I forgot ent_setvetex uses dx coordinate system so z and y are swapped. I was getting really odd results till I realized and simply setting the parameters manually changing the axsis solved everything:
c1.v.nx=mid_nml.x;
c1.v.ny=mid_nml.z;
c1.v.nz=mid_nml.y;

The corners get more complicated though, here is an example of one of my corner functions:
Click to reveal..
void avg_normals_SE(int x, int y)
{
int tv; //vector counter
VECTOR nml[4];
VECTOR mid_nml; //average height of touching vertex's
int nml_count;
CONTACT* c1; //contact vertex for Terrain


vec_for_normal(nml[0],Terrain[x][y],1089);
nml_count=1;
if(x<TERRAIN_COUNT-1)
{
vec_for_normal(nml[1],Terrain[x+1][y],1057);
nml_count++;
}
if(y>0)
{
vec_for_normal(nml[2],Terrain[x][y-1],33);
nml_count++;
}
if(x<TERRAIN_COUNT-1 && y>0)
{
vec_for_normal(nml[3],Terrain[x+1][y-1],1);
nml_count++;
}

if(nml_count>1)
{
vec_set(mid_nml,nullvector);
for(tv=0;tv<nml_count;tv++)
{
vec_add(mid_nml,nml[tv]);
}
vec_normalize(mid_nml,1);

c1 = ent_getvertex(Terrain[x][y],NULL,1089);
//vec_set(c1.v.nx,mid_nml);
c1.v.nx=mid_nml.x;
c1.v.ny=mid_nml.z;
c1.v.nz=mid_nml.y;
ent_setvertex(Terrain[x][y],c1,1089);

if(x<TERRAIN_COUNT-1)
{
c1 = ent_getvertex(Terrain[x+1][y],NULL,1057);
//vec_set(c1.v.nx,mid_nml);
c1.v.nx=mid_nml.x;
c1.v.ny=mid_nml.z;
c1.v.nz=mid_nml.y;
ent_setvertex(Terrain[x+1][y],c1,1057);
}
if(y>0)
{
c1 = ent_getvertex(Terrain[x][y-1],NULL,33);
//vec_set(c1.v.nx,mid_nml);
c1.v.nx=mid_nml.x;
c1.v.ny=mid_nml.z;
c1.v.nz=mid_nml.y;
ent_setvertex(Terrain[x][y-1],c1,33);
}
if(x<TERRAIN_COUNT-1 && y>0)
{
c1 = ent_getvertex(Terrain[x+1][y-1],NULL,1);
//vec_set(c1.v.nx,mid_nml);
c1.v.nx=mid_nml.x;
c1.v.ny=mid_nml.z;
c1.v.nz=mid_nml.y;
ent_setvertex(Terrain[x+1][y-1],c1,1);
}
}
}


Hope this helps and saves you headache and time for when you get to that part in your version.
You are not alone either grin

And thanks again for pointing me in the right direction!

Your LOD idea is pretty good though im not sure I will implement something like that. I am going for a cartoony/goofy look to my game. Mainly for 2 reasons, its more friendly for an indie's time budget, and because with simpler models I can load tons more. My trees will probably be a deformed low poly cylinder for the stump, and either a deformed low poly cone or sphere for the leaves. My trees will probably have less polygons than a low LOD tree normally has in one branch.


Uploaded with ImageShack.us
(and I'm considering lowering the poly count even more, LOL!)


"The more you know, the more you realize how little you know..."

I <3 HORUS
http://www.opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=401929&page=1