1 registered members (Ayumi),
1,170
guests, and 2
spiders. |
Key:
Admin,
Global Mod,
Mod
|
|
|
[Solved] Multi-threading (for A8) guru needed.
#384491
10/03/11 23:09
10/03/11 23:09
|
Joined: Feb 2008
Posts: 3,232 Australia
EvilSOB
OP
Expert
|
OP
Expert
Joined: Feb 2008
Posts: 3,232
Australia
|
Hiya guys.
Im working of a proof-of-contept project at the moment, and it seems to me that multiple threads would be advantageous.
Im looking into having the game spawn a couple of worker threads to better time-manage some of the more time-intensive terrain handling stuff, maybe more will be added later.
Specifically(at this time): ONE: Loading a chunk of data (using WinAPI's) from a large file on HDD and inserting it into a selected memory locations. TWO: Taking data from an array filled by thread ONE, adjust the heightmap of a 128x128(max) terrain slab.
These threads operate independantly, and adjust a flag-system stored within the data-chunk they are manipulating.
ie: The first byte of the memory-area filled by thread ONE is a status flag, showing when 'busy' and when 'complete'. And thread TWO sets a flag of the entity to on WHILE it is processing.
So I am being careful (so far) that the threads never CREATE or REMOVE any objects, and the objects that the data is in can see if its 'busy' and pause its removal (either the data or itself) until the processes are complete.
So now you know what Im tring to achieve, here comes the "ask". Im not asking for it to be written FOR me, I feel that defeats my purpose. I just need some functional code, showing how to get a single (or even multiple if tricky) worker-thread up and running, and how to safely manage him/them. (including terminations/sleeping/etc later on) And hints and tips ion threads in general. ... And someone to ask questions of when it all goes horribly wrong.
Thaks guys...
"There is no fate but what WE make." - CEO Cyberdyne Systems Corp. A8.30.5 Commercial
|
|
|
Re: Multi-threading (for A8) guru needed.
[Re: HeelX]
#384495
10/03/11 23:41
10/03/11 23:41
|
Joined: Apr 2007
Posts: 3,751 Canada
WretchedSid
Expert
|
Expert
Joined: Apr 2007
Posts: 3,751
Canada
|
Well, manipulating a terrain is only possible on the mainthread since Gamestudio doesn't impose any kind of locking mechanism on its objects. However, you can load your data and process it in a secondary thread and unlike HeelX stated this makes sense for cases where you want to load the data without blocking the main thread. Now about synchronizing, this is done by using mutexes. A mutex (or mutual exclusion) is an object that allows only one thread to pass execution and let the other threads wait. What you want to achieve can be done by creating one mutex and one state variable. Whenever you want to read or write from/to the variable you first lock the mutex to ensure that only one thread can alter the state variable. So, here is some pseudo code for your main thread:
void checkState()
{
lock_mutex(globalMutex); // Lock the mutex so that only we can read it
if(globalState == allowReading)
processLoadedData(); // If the other thread signaled that we can read the data, read it.
unlock_mutex(globalMutex); // Unlock the mutex so other threads can lock it.
}
And here is the one for your secondary thread:
void loadData()
{
lock_mutex(globalMutex); // Lock the mutex
globalState = waitForData; // Mark the state as "we are currently busy reading"
unlock_mutex(globalMutex); // And then unlock the mutex so that we don't block other threads
readDataFromDisk(); // Read the actual data from disk
// And now set the state to ready to read.
lock_mutex(globalMutex);
globalState = allowReading;
unlock_mutex(globalMutex);
}
So, now for the actual functions you need to call: Mutexes:
HANDLE myMutex; // Handle to a mutex
myMutex = CreateMutex(NULL, FALSE, NULL); // Default security attributes, not initially locked, no name
CloseHandle(myMutex); // Destroys the mutex
WaitForSingleObject(myMutex, INFINITE); // Locks the mutex and waits infinite for it
ReleaseMutex(myMutex); // Unlocks the mutex
Threads:
HANDLE myThread; // Thread handle
myThread = CreateThread(NULL, 0, myFunction, NULL, 0, NULL); // Default security attributes, default stack size, myFunction should be the entry function, no parameter passed to the function, default flags and we aren't interested in the id of the thread (otherwise we had to pass a pointer to LPDWORD variable
DWORD myFunction(void *ignored)
{
// Do something in the secondary thread
return 0; // Signal succes. If your thread failed, return 1.
}
I can highly recommend you the MSDN pages for all of the above functions, they have all the details about possible parameters and links to related functions that might help you too. Last but not least, use the C FILE functions (fopen(), fread(), fclose()) to read your file as the Gamestudio file_ functions might not work in a secondary thread.
Shitlord by trade and passion. Graphics programmer at Laminar Research. I write blog posts at feresignum.com
|
|
|
Re: Multi-threading (for A8) guru needed.
[Re: WretchedSid]
#384497
10/04/11 00:22
10/04/11 00:22
|
Joined: Feb 2008
Posts: 3,232 Australia
EvilSOB
OP
Expert
|
OP
Expert
Joined: Feb 2008
Posts: 3,232
Australia
|
HeelX: Blame this on my over-simplification. Thread TWO will be working on an array of data that is SOMETIMES replaced by thread ONE. Thread TWO will be re-reading (never writing) the array several times per update by thread ONE.
And both thread ONE and thread TWO both have many arrays to manage. So they will never (and I WANT it like this), never be working on the same one at the same time. And I only want one thread each for these two tasks, because they are either time-hungry(ONE) or CPU-hungry(two). So by buffering data 'to be processed' by these threads, it lets me create a 'known' bottleneck that will allow me to limit CPU/HDD utilisation by these tasks.
JustSid: Only just starting to read yours now...
[PS] They are actually models PRETENDING to be terrains. And they will actually be hidden (INVISIBLE) during the vertex-adjustment phase, then swapped into replace the old one.
Last edited by EvilSOB; 10/04/11 00:31. Reason: PostScript
"There is no fate but what WE make." - CEO Cyberdyne Systems Corp. A8.30.5 Commercial
|
|
|
Re: Multi-threading (for A8) guru needed.
[Re: EvilSOB]
#384499
10/04/11 00:40
10/04/11 00:40
|
Joined: Feb 2008
Posts: 3,232 Australia
EvilSOB
OP
Expert
|
OP
Expert
Joined: Feb 2008
Posts: 3,232
Australia
|
JistSid: Now Ive done a 'quick' readthrough of your post, but is using mutex's necessary? (in this situation anyway) Read my above reply to HeelX and you'll see my more exact needs. TO MY MIND, by using the status-flags embedded in the data-area, I am really creating a 'different' mutual-exclusion system. I prefer mine cause the original GS thread can see the states easily too. It needs to react according to those statuses.
Note this is only from a QUICK read-through. Im now moving on to a PROPER reading.
[EDIT] I now have a better understanding, but still dont see a need for mutex's. Feel free to argue though... Im in the midst of uilding a semi-functional example to test with. I'll post it when Im finished it, or stuck. Question: You suggested using fopen fclose, etc. I was going to use the CreateFile API and its fellows. Any reasons against?
And thanks guys. Any input is good input. Or so my missus says. But she lies alot...
Last edited by EvilSOB; 10/04/11 06:04. Reason: edited in extra notes and a question
"There is no fate but what WE make." - CEO Cyberdyne Systems Corp. A8.30.5 Commercial
|
|
|
Re: Multi-threading (for A8) guru needed.
[Re: EvilSOB]
#384506
10/04/11 06:04
10/04/11 06:04
|
mercuryus
Unregistered
|
mercuryus
Unregistered
|
Hi EvilSOB, once apon a time (2 1/2 years ago) i posted a small and very easy example to use multithreading with lite-c. My solution seams to be the better for your idea. Unfortunately the example got lost from my server - i will upload the example again (in ~ 10h) - and pm you the link... boost up your game...
|
|
|
Re: Multi-threading (for A8) guru needed.
[Re: ]
#384507
10/04/11 06:06
10/04/11 06:06
|
Joined: Feb 2008
Posts: 3,232 Australia
EvilSOB
OP
Expert
|
OP
Expert
Joined: Feb 2008
Posts: 3,232
Australia
|
2 and a half years !?! No wonder there wasnt much in the ubb-search function...
Thanx
"There is no fate but what WE make." - CEO Cyberdyne Systems Corp. A8.30.5 Commercial
|
|
|
Re: Multi-threading (for A8) guru needed.
[Re: EvilSOB]
#384520
10/04/11 10:59
10/04/11 10:59
|
Joined: Feb 2008
Posts: 3,232 Australia
EvilSOB
OP
Expert
|
OP
Expert
Joined: Feb 2008
Posts: 3,232
Australia
|
OK guys, my brain is wearing thin, the sandman is calling, but he knows I have a shiv! Here is the example code I said I was writing that, I feel, does everything I want. Be warned here and now, it WILL compile and run, but the threads wont get created... I currently have the CreateThreads remarked out, as I dont yet have a data file to test with. But Im posting the code so if anyone can see a problem with my logic or handling OF THE THREADS just by reading though my (rather messy but fairly well documented) code, then PLEASE point it out for me! I'm going to try and generate a faked-up data file now IF I can dodge the sandman long enough.... He's SNEAKY! Thanks again all... It may look ugly here, but in SED it is much more readable
#include <acknex.h>
#include <default.c>
#include <windows.h>
//
//
//---------------------------------------------------------------------------------------
//
HANDLE Thread_ONE; DWORD worker_ONE(void *ignored); short* worker_ONE_data=NULL;
HANDLE Thread_TWO; DWORD worker_TWO(void *ignored); short* worker_TWO_data=NULL;
ENTITY* worker_TWO_entity;
long close_threads = 0; //global thread terminator
//
//
//---------------------------------------------------------------------------------------
//
#define HeightMap "height_map.dta"
#define HeightMapWidth 16384
#define HeightMapHeight 12288
#define TerrainDimensions 128
//
//---------------------------------------------------------------------------------------
//
//
void main()
{
wait(1); level_load(NULL); wait(1); diag("\n\n\n");
draw_textmode("Times",0,32,100); warn_level = 3; on_esc = NULL;
//------------------------------------------------------------------------------------
//
// Thread_ONE = CreateThread(NULL, 0x00, worker_ONE, 0x00, 0x00, 0x00);
// Thread_TWO = CreateThread(NULL, 0x00, worker_TWO, 0x00, 0x00, 0x00);
//
//
while(!key_esc)
{
wait(1);
}
//
close_threads = 1; // game over, time to kill the threads
while(worker_ONE_data != Thread_ONE) wait(1); //wait until thread ONE terminated
while(worker_TWO_data != Thread_TWO) wait(1); //wait until thread TWO terminated
//
//
//
//------------------------------------------------------------------------------------
if(key_shift) exec("notepad.exe", "acklog.txt");
sys_exit(0);
}
//
//
//
//---------------------------------------------------------------------------------------
//
// The below file-retrieval code "largely" conjectural, as the files design not finalized
// and this code has no real error-checking and it aint optimised by a long shot...
DWORD worker_ONE(void *ignored)
{ //NOTE: for the record, THIS code is limited to 1.2g image files, not the API's
var last_frame = 0; //place to remember last frame I did processing on.
short* data_block = NULL; //pointer to data block supplied via worker_ONE_data
long file=0, y, x_off,y_off, count, file_ptr_lo; //various local workspaces
long width=sizeof(short)*TerrainDimensions; //various local workspaces
Sleep(0x1000); //snooze for a bit. Let everything else get running.
//
while(!close_threads) //loop until global terminate is triggers
{
if((worker_ONE_data)&&(total_frames>last_frame))
{
data_block = worker_ONE_data; data_block[1]=1; //set status active
x_off=(long)data_block[1]; y_off=(long)data_block[2]; //retrieve offsets
if((x_off<0)||(y_off<0)) { error("poo! Illegal Offsets!"); break; }
SetFilePointer(file, file_ptr_lo, 0x00, 0x00);
file_ptr_lo = x_off + (y_off * HeightMapWidth);
file = CreateFile(HeightMap, 0x80000000, 0x00, 0x00, 0x03, 0x00, 0x00);
if(!file) { error("Oh crap! Cant find HeightMap!"); break; }
//start retrieving data
for(y=0; y<TerrainDimensions; y++)
{
ReadFile(file, &(data_block[y*TerrainDimensions+1]), width, &count, 0x00);
if(count!=width) { error("Dammit! Hit end-of-file!"); break; }
file_ptr_lo = x_off + ((y+y_off) * HeightMapWidth);
SetFilePointer(file, file_ptr_lo, 0x00, 0x00);
}
CloseHandle(file); file=NULL;
data_block[0] = 0; data_block = NULL; //set status to complete.
worker_ONE_data = NULL; //disconnect from data source/s
last_frame=total_frames; //remember to not process more THIS frame
}
//
Sleep(100); //snooze for 100 milliseconds (for now, will drop to 10 or 1 or zero)
}
worker_ONE_data = Thread_ONE; //signal that this thread is terminated
}
//
//
//---------------------------------------------------------------------------------------
//
// The below vertex-adjustment code "largely" conjectural, as this code has only been
// tested to COMPILE state. MUCH testing will be needed to verify thread-safety...
DWORD worker_TWO(void *ignored)
{
var last_frame = 0; //place to remember last frame I did processing on.
short* data_block = NULL; //pointer to data block supplied via worker_TWO_data
ENTITY* entity = NULL; //pointer to data block supplied via worker_TWO_entity
var x, y; //various local workspaces
CONTACT c; D3DVERTEX* verts; //various local workspaces
Sleep(0x1000); //snooze for a bit. Let everything else get running.
//
while(!close_threads) //loop until global terminate is triggers
{
if((worker_TWO_data)&&(worker_TWO_entity)&&(total_frames>last_frame))
{
data_block = worker_TWO_data; data_block[1]=1; //set status active
entity = worker_TWO_entity; //retrieve pointer to affected terrain entity
ent_getvertex(entity,c,1); ent_setvertex(entity,c,1); verts = c.v;
//start modifying terrain heights
for(y=0; y<TerrainDimensions; y++) for(x=0; x<TerrainDimensions; x++)
verts.y = data_block[y*TerrainDimensions+x+1];
data_block[0] = 0; data_block = NULL; //set status to complete.
worker_TWO_data = worker_TWO_entity = NULL; //disconnect from data source/s
last_frame=total_frames; //remember to not process more THIS frame
}
//
Sleep(100); //snooze for 100 milliseconds (for now, will drop to 10 or 1 or zero)
}
worker_TWO_data = Thread_TWO; //signal that this thread is terminated
}
"There is no fate but what WE make." - CEO Cyberdyne Systems Corp. A8.30.5 Commercial
|
|
|
Re: Multi-threading (for A8) guru needed.
[Re: EvilSOB]
#384535
10/04/11 16:24
10/04/11 16:24
|
mercuryus
Unregistered
|
mercuryus
Unregistered
|
dont know if you still need it but you shouldn't use any ent_ functions or in a thread!You can store the new vectors for the vertics in a matrix (array) with one thread - and when done for all (flag?) you can call a function in your main program loop to set them to the terrain.
|
|
|
Re: Multi-threading (for A8) guru needed.
[Re: ]
#384536
10/04/11 17:15
10/04/11 17:15
|
Joined: Apr 2007
Posts: 3,751 Canada
WretchedSid
Expert
|
Expert
Joined: Apr 2007
Posts: 3,751
Canada
|
Here is another example for background loading of assets, its a bit more complete than mercuryus' and also shows the usage of mutexes (and yes, they are important as you want atomic read, since you can't gurantee that one thread alters the state while you read it (or vice versa)!) http://cl.ly/6wN1Btw, as for "you shouldn't use any ent_" functions: You MUST NOT! use any engine function at all in a secondary thread (you can use them on the main thread though). Like I already wrote, gamestudio doesn't impose any kind of locking for its objects so that there is no way to synchronize two or more threads.
Shitlord by trade and passion. Graphics programmer at Laminar Research. I write blog posts at feresignum.com
|
|
|
|