LOD, gigantic Terrains

Let us take for example Skyrim, just how big the world is!
Now I was wondering how you could realize huge worlds areas with the acknex engine without killing fps or crashing the game because there are no nexus left etc...

We would have to reduce more and more details the further the camera goes away and force to only show a maximum amount of entities etc., in other words a LOD system.

That is the same trick these guys are using:
Youtube

Did anybody already worked on that, will or is it a feature for the acknex engine?
In other words, does the acknex engine support any kind of LOD?
And will it be improved if it is already available?

Greetings Chuck

You can use Chunked Terrain. But this is also very limited.

Currently I'm working on real unlimited terrain. And by unlimited I mean unlimited (the only limiting factor would be HDD space, but that's not a problem on modern PCs).

My trick is, the whole world (terrain) is splitted into tiles. Each tile is saved on HDD in a custom file format where vertex position data (only Z position) is saved and maybe the normals too. And the texture as external file.

Now I create only a bunch of unique terrain tiles on level loading via ent_clone. These unique terrain tiles will be updated with the tile data saved on HDD. For example: you have a really huge world consist of 10,000 terrain tiles. If you would load them all via ent_create or ent_morph into the engine, the nexus limit would be reached very quickly. Instead there are just, for example, 32 unique terrain tiles, as much as the camera clipping distance allows to show.

Now, if the player moves forwad, the tiles behind the camera, which moves away from the player, are "freed". This means the texture will be remove from video memory via ent_purge, and the ENTITY* of this unique tile will be available for the tile which comes closer to the player. Now this tile will be filled with the vertex data which we saved on HDD and load the new texture on this terrain tile entity.

The BIG advantage by reading vertex data from HDD is, no new Nexus memory is needed. So you can do unlimited terrain.

The biggest disadvantage is, that filling the terrain tile with the vertex data consums processor speed and therfore FPS. But this can be reduced to a minimum by using small tile sizes and by filling the vertex data of a tile not in a whole frame, instead splittng the vertex data reading into several frames. This will keep the FPS high and the player won't notice the loading.

Another problem is the "var" limit of -999.999 to 999.999. For this problem I don't have tested any solution yet. But one approach would be, to define zones of your world, if the player walks into a new zone, the whole level will be shifted in X and Y direction (this should be really fast, maybe the user will only notice a framerate drop for less than a second).

I hope this was understandable

I am facing the same challenge when it comes to terrain LoD. Allot of things are bothering me when it comes to implementing this feature. I'm limited to -2000000 to 2000000 quants in world size. I need to save poly and using large poly size with detail map texture makes the terrain polygon shake so I have to eliminate the detail texture and use plain texture instead. I wanted to use my own terrain LoD but loading different LoD for each tile is slowing things down and I also have to work with each node's texture Lod. Haven't tried loading z position of each vertex yet but I'm wondering how to use it in a way that it would line well with other tiles.

Then after solving the terrain LoD problem you still have to figure out how to load and unload terrain entities which is I'm trying to figure out with the terrain problem.

These are both pretty naive approaches, and unlikely to perform well in a real environment. If I had to take a guess, I would say Oliver2s' method is currently performing good because he is hitting the filesystem cache pretty hard (can you try to flush it completely and then retry your code?)

The problem is that HDDs are inherently slow and it may take up to two complete rotations of the platters until the head is over a chunk of data you want to read, and that assumes that the HDDs is completely spun up already and not serving anything else. Just loading on demand won't cut it, you should look into pre-loading chunks that are not yet visible, but in vicinity to already visible chunks, to avoid doing the HDD roundtrip in a blocking manner (as in: Read them in the background).

And the file format is also important, it may make sense to duplicate data in a way that allows you to just read data in sequence instead of having to seek around all the time. The Just Cause 2 guys ended up writing a system that uses multiple layers of terrain detail with duplicated data that allows them to just read in a huge chunk of the file at once as the player moves through the world. Their post mortem may give you some ideas.

I like Oliver's approach, but another problem could arise when it becomes necessary to load a lot of objects corresponding to each terrain tile (vegetation, buildings etc.).
I think high nexus usage can be avoided by using sys_malloc/sys_free, but they are also time consuming operations...

Sure. But that's a compromise between speed and size of level. I could preload everything into Nexus. But as we all know, this very limited in case of big levels. So loading data from HDD is a good compromise for me.

Perhaps you may check out my Neomill Project "Tradmill" script in the C-Script/WDL section. Its written in C-Script/wld but rather simple and should be understood easily and could be translated into Lite C quickly.

Once you understood how the Neomill treadmill script works you could improve the scipt building your own LOD Lite-C script for terrain tiles.

You may need to change the 9 Tile Treadmill into a 16 tile treadmill or with even more tiles.

Let the tiles far away from you loaded in as simple low poly mdls or hmps and the terrain you are on or nearby in high poly quality.

The only disadvantage of my scripted mill is that all tiles must be files and therefor are visible for the Gamer.

Anyway, check ouut it if you haven't alread. It can be downloaded from the acknex resourece download section (code section)

Greets

J. Hunke

I'm still testing my treadmill script I explained above. The data (height of each vertex) can also be loaded into RAM and therefore don't needed to be loaded from HDD all the time. Only the textures need to be loaded at runtime from HDD (as I explained above too). Loading vertex height data into RAM would consum around 40 MB of memory with 20,000 tiles. In my world scale that would be an area of 576 km².

But as sivan mentioned. There's still the problem with unique entities on the terrain (buildings, vegetation, etc..) which increases the nexus.

Hence why I said streaming it in in the background. Or just use asynchronous I/O, and you don't even have to touch the threading stuff.
Seriously, try it with a cold filesystem cache and see if it's still a good trade-off to just load on demand.

Yes, I'm still testing...

@Oliver: What about using vertex texturing for the texture problem?
Instead of having a blendmap use the uv-coords or other data of your terrain to blend the different textures.
This should be faster to load (could be loaded with the normal terrain data) and rendering as well (- 1 texture-fetch)

The resolution would not high enough. My terrain has a resolution of 32x32 vertices. But the blendmap is 128x128 px.

Oliver2s, your methodik sounds quite impressive.
But let us say there are suppose to be villiges at surtain areas of the world, how would you do that?

So what I am looking for is to create alot of islands (6-7, you sail to them over the ozean per boot) which each have 3-5 of the the size of the landscape terrain (3dgs car templer demo).
That should actully be passable with the acknex engine.

You mean to avoid a nexus overflow?

Yes.
While you are on one quite big terrain, the otheres which are far away (you have to travel per ship) just are (with everything on those terrains) passable and invisible until you reached a surtain distance.
In that way you could make a really big world with alot of "countries" and towns to discover.

The questin is, how big can a terrain be without killing fps and nexus?
5 times as big as that terrain from the car demo would be great!

You just put more information in the level format and stream that from disk? I don't quite see the problem, loading terrain chunks on the fly isn't much different than loading models on the fly and placing them in the world.

its not about scale, but about how much details and objects you have

Exactly. Hence why you want the streaming in the first place.
I assumed that was the premise of this thread?!

maybe this can help (libAsset by JustSid) (I have not tried it but will soon): http://www.opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=427781&page=1