I had the very same idea for an infinite terrain system (endless world) I came up with and am running into the same problems...

I load a certain number of terrains around the player and I also load a heightmap to terrains at runtime depending on the character's position. I first tried to move them (like your first attempts) but they cannot move. So I started to remove the ones left behind and create the new ones the player is coming close to...

Then I found this thread and it is like a slap on the face, lol!

If I understood you correctly, you are trying to achieve the same thing as I am? And the old entities being removed never free the memory? So I will end up running out of memory even though I always only have 25 (5x5) terrain entities loaded at once?

Did I understand everything correclty? even when removing entities the memory usage just keeps growing and growing?
So then your solution is to make huge models with lots of vertices that look like a terrain so you can use them instead and move them around?

This does not seem like a viable solution to me if all other entities also behave this way (I don't know how your game works, if your game is empty and only uses terrains and a player model it might work fine. But in mine I need to load more things on the terrains).
If all models behave this way with memory, this brings another concern for me: what about other types of entities? (my plants, enemies, and buildings are the frist examples that come to mind...)
Even if I start using entities to simulate terrains like you do, so I can move them and never have to remove/create them all the time, won't the other entites (enemies, plants) that are also being created/removed leave behind alot of unused memory?

I am not sure if I understood everything correctly, but if I did... Then even if I fix the memory problem for terrains like you did (using huge models and moving them), everything else that gets leaded/removed will still produce this memory problem?

What are your thoughts on this new problem? Have you already thought about it? Do you have a solution for this also?

Yes.
Not all of it.
Eventually.
Yes.
Yes.
Yes.
Me too.
They will cause the same problems too.
Sure will.
Certainly.
I hate it!
Many months of it.
Maybe.



BTW, a more detailled post will follow soon...

Hope this helps clarify things a bit...

YOU ... ARE ... NOT .. ALONE

I was thinking about this after I posted and I came up with a terrible solution (but that has some similarities with yours): loading on startup a certain number of each type of entity that will be repeated alot on the terrain. For example load 100 of tree1.mdl, another 100 of bush5.mdl, 100 of tree3.mdl, etc, etc etc... and moving them all to somewhere well hidden from (like lower than any terrain (vec_set(my.x, vector(0,0,-1000000))). whenever I need a tree instead of ent_load I just set its pos to the correct location, and instead of deleting one I just set it back to (0,0,-100000000). But I didn't really like this idea and it seemed like a very sloppy workaround with many limitations.

After reading your answers, ent_morph + storing unused entities seems like a very smart approach to this problem (and much better than my terrible&&painfull idea).

I have been thinking about this for a while and come up with a another concept to fix some of the problems both systems have. My idea has severe limitations. Your approach of loading a ship-load of entites + ent_morph for re-use is much better but also has certain limitations and a potencially extreme load time.

In another aspect of my game I have been working alot with malloc() sizeof() and free() for several linked lists, and this is what gave me the idea. We create an empty linked-list on startup and program our own ent_create_2 and ent_remove2 functions.

The linked list would just be a struct of 2 pointers, one entity pointer, and one struct pointer.

The ent_create_2 function first looks at the linked list, if it is empty it creates a new entity with our position and model parameters. else if there is already an item in the list, it modifies it position properties and uses ent_morph with the model parameter, and removes the item from the linked list.

The ent_remove_2 function sets the model's invisible and passable flags, morphs it to a very simple dummy model (one centered vertex), places it in a hidden location well under our terrains (0,0,-1000000000), and add the item to the linked list.

with this system you avoid having to create a ship-load of entites on launch, saving alot of load time. it only increases the amount of entities loaded when needed (you reach an area with more entities than the previous areas you visited). and in the worst-case scenario you will never have more entites loaded (used + not_used) than the zone you visited that had the most entities. This system also solves the problem of not knowing how many entities you will need before reaching a certain area.

My only concerns now are the free() and ent_morph() instructions... Seeing how ent_remove() works I am now scared that any other removal/modification instruction may also leave behind unused memory. But even though ent_remove() is terrible, if these other two instructions do work how they are supposed to we might have solved the problem with free() and ent_morph(). Do you think it is worth a try? Do you know if free() and ent_morph() also leaves behind unusable memory?

If this system works I will probably make it a separate module and include it in every future project and never ever use ent_remove again. (even on projects that dont have infinite worlds! After reading this thread I have developed a passionate hatred of the ent_remove function!)

The only problem I see with your approach (and also my linked-list approach) is that action functions cannot be used with entities using this system (and that potencially also includes events).

I will post back in a while with a modified version of my current linked-list system adapted for this problem.

EDIT: You made me laugh with this answer when re-reading my initial post:

This is the entity system I talked about above that should solve our problems (or at least most of them). I have to leave and it is untested, but it should work. If it dosn't work I will revise it when I come back home later tonight or tomorrow.



The code is slightly commented, but if there are any questions feel free to ask.

I hope this helps us!

If you come up with any problems derived from using a system like this for recycling entities please post and I will think of another solution.

The only thing I see failing with this is that entity actions are not supported for recycled entities, but for trees, buildings, and most static objects it shouldn't be a problem. If you come up with a better version/system to support actions and events for recycled entities feel free to modify, and mabe post your ideas so I can implement also, I am very interested in giving this system as much flexibility as possible to never have to use ent_remove ever again! grrrr (IMHO, if entities leave some unused memory behind when removed... then this is how ent_create and ent_remove should have been made from the start)

After looking back over my old code, which I havent played with in quite a few of months,
I found my "partially" developed linked-list system that I was going to use with my terrains...
Since then Ive gone for an array(grid) with a fixed number of pre-created entities.

My old stuff has MUCH in common with yours, including a NOTATION suggesting more research/testing
of an "ent_create as needed with limit" approach. Sound familiar?
Some other 'notes to self' were as follows.

Dual linked lists. Identical structure, one is 'active' other is 'spare'. Entity moves from list to list as its status changes.

Dual-priority ent_recreate [you called yours ent_create_2]. Dual priority calling.
HIGH priority = use first found 'spare' entity. LOW priority = search list to try to find matching model to requested one.

ent_idleize (your ent_remove_2) PASSABLE, INVISIBLE, Z=-5000, kill any actions.


And thats just a few.

I too am concerned about ent_morph, and because I normally need to use ent_clone alot, I worry about it too.

With your usage of Malloc, Free etc, you should shift to SYS_malloc and SYS_free etc.
More engine friendly, and they DO release all memory for re-use...

ALSO!!! BEWARE OF sizeof.... check my posting in the BugHunt forum.


Actions are not a problem. But they are 'faked' actions. Like so...

And you just call it from inside the ent_recreate function, with the 'new' entity as the parameter.

And inside the entities action-loop, just watch to see if "I" am in the ACTIVE linked-list,
or in the IDLE linked-list. If I am in the IDLE list, just execute a RETURN;
I would never do something this 'dirty', but you get the idea.


But Im not actively working on this ATM... A few sub-projects stand in the way.

Im helping Esper out with his menu system, and I've got some outstanding weapon-system stuff for 3run I gotta get to,
and then I have to take a look at my ancient port-IO contribution for someone...

Busy, busy, busy...

The second linked-list for stopping the action feels like overkill to me. It seems to complicate the linked lists alot to controll something that dosn't need a second list. It would probably be easier to set a skill to 1 when creating/recycling an entity, and setting that skill to 0 when adding to the "inactive list", then in the function that acts like a "fake action" instead of using a while(1) and a return just use a simple while condition that uses that skill.

This would remove the need for a second linked-list, and it would also avoid the need for the linked lists to be dual (or bi-directional) for searches. That way you could keep the list working like a simple stack as in my previous post's example.

About the actions: I see how a function recieving a ent pointer could substitute an action. But how do I tell my ent_create_2 function which function it needs to run for the ent? is there any way to set a pointer to a function and pass that as a parameter to my ent_create_2()? Or to use a string containing the funtion name to use some macro to run a function?

About the events: I havn't tested it but it seems with that function acting as an action it should be possible to set the events and all... But how would I invert the process later when recycling the entity to remove its events? the my.event could be easily redirected to an empty "dummy" event function and reset later to whatever new event function the ent needs. But how would I know what flags were previously enabled to disable them? (like ENABLE_BLOCK or ENABLE_ENTITY)



EDIT: Being so busy is a good sighn. It means 1.You enjoy what you do, and 2.you are good at what you do

The IDLE link-list does more than JUST allow the action to self-terminate.
It is also an INSTANT way to grab an idle entity. Just grab the first in the list.
That way you dont need to wade through potentially dozens of active entities
when trying to find an idle one to re-use.
And (from memory) it was only like 3 lines of code to make an entity switch lists.
The IDLE list was only implemented single-directional searches. Thats all it needed.


How do you NORMALLY tell ent_create what action to use? Do the same for ent_create_2!


As for the events... have the funct/action ITSELF shut them down. Put the event-disabling
code AFTER the while loop. So when the entity recognises it is idle, and exists the loop,
it will then disable its own events...

I have been reading sys_malloc and sys_free in the manuall but I don't uderstand what it does. Why is it different/better?


wow, I had no idea functions could behave like a pointer. I thought it would be much more complicated to get that working. Thanks alot for the info!
I also have a uni directional linked-list for the "inactive" entities, and use it as a stack for pulling the last one for re-use. When I said mabe two linked lists might not be necessary I meant the other one, the "active" linked-list.

And now that I am using models for my terrains I am having trouble with the shadows... It looks terrible where they join. Did you have the same problem? How did you fix it?
It looks like its a problem with smoothing the gourad shading on each model individually.
I tried with the UNLIT flag and setting its ambient high so it would be lit. But it seems I cannot get rid of its gourad shading that way.

Here is link to a quick video I just made showing how the edges of the terrains are shading separatly for each model:
http://www.youtube.com/watch?v=emOdg111H4Q

I tried with setting unlit, cast, and also resetting the shadow flag. But it still looks awfull, I can't think of how to get rid of those terrible shadows that make the edges of my terrains so obvious...
Any suggestions?

Well, functions and pointers are really the same.
An ACTION (and also an EVENT) are really just VOID function, and by passing its address
as a parameter is as easy as passing any other VOID*. Then assign that address
to an unused function pointer... easy.


I see your shadowing problem, and I EXPECT to hit the same, but Im not that far yet...
Im still trying to get the entities to arrange themselves properly around the focus point.
Not so easy because I have a circle of LOD_0 at the focus, surrounded by a thick
ring of LOD_1 entities, etc. out to LOD_3. Getting them to arrange based on a
single formula is the tricky bit, as I need a single formula to to this .
Because the game (at boot up) calculates the clip_far to use, and how "big" each lod circle is.
This result gets fed to the terrain-management, and the entities are created/arranged.
So... for low-power PC's, the total terrain "circle" is small (115 entities),
and for higher-powerd PC's it is bigger (465 entities).
[/i]See spoiler at the bottom for some preliminary calculations.[/i]

Ive not gotten to changing heights yet. Once the entities are arranging correctly,
then I'll implement the height mapping. So I expect your problem, even though
I havent had to deal with it "yet".

What I expect to need to do is integrate a "normal recalculation" into the stitcher code
that looks at BOTH entities, and adjusts BOTH their normals for the edge pixels/polys,
based on the value one entity normal plus the other entity normal divided by two.



My entity LOD rings. (still in preliminary testing ATM)
This is only showing THREE of the seven steps possible.