Gamestudio Links
Zorro Links
Newest Posts
Executing Trades on Next Bar Open
by vicknick. 06/13/24 08:51
Zorro Beta 2.61: PyTorch
by jcl. 06/10/24 14:42
New FXCM FIX Plugin
by flink. 06/04/24 07:30
AlpacaZorroPlugin v1.3.0 Released
by kzhao. 05/22/24 13:41
Free Live Data for Zorro with Paper Trading?
by AbrahamR. 05/18/24 13:28
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
1 registered members (Ayumi), 1,170 guests, and 2 spiders.
Key: Admin, Global Mod, Mod
Newest Members
AemStones, LucasJoshua, Baklazhan, Hanky27, firatv
19059 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 1 of 2 1 2
[Solved] Multi-threading (for A8) guru needed. #384491
10/03/11 23:09
10/03/11 23:09
Joined: Feb 2008
Posts: 3,232
Australia
EvilSOB Offline OP
Expert
EvilSOB  Offline OP
Expert

Joined: Feb 2008
Posts: 3,232
Australia
Hiya guys.

Im working of a proof-of-contept project at the moment, and it seems to me that multiple threads would be advantageous.

Im looking into having the game spawn a couple of worker threads to better time-manage some of the
more time-intensive terrain handling stuff, maybe more will be added later.

Specifically(at this time):
ONE: Loading a chunk of data (using WinAPI's) from a large file on HDD and inserting it into a selected memory locations.
TWO: Taking data from an array filled by thread ONE, adjust the heightmap of a 128x128(max) terrain slab.

These threads operate independantly, and adjust a flag-system stored within the data-chunk they are manipulating.

ie: The first byte of the memory-area filled by thread ONE is a status flag, showing when 'busy' and when 'complete'.
And thread TWO sets a flag of the entity to on WHILE it is processing.

So I am being careful (so far) that the threads never CREATE or REMOVE any objects, and the objects that the data is in
can see if its 'busy' and pause its removal (either the data or itself) until the processes are complete.


So now you know what Im tring to achieve, here comes the "ask".
Im not asking for it to be written FOR me, I feel that defeats my purpose.
I just need some functional code, showing how to get a single (or even multiple if tricky) worker-thread up and running,
and how to safely manage him/them. (including terminations/sleeping/etc later on)
And hints and tips ion threads in general. ... And someone to ask questions of when it all goes horribly wrong.


Thaks guys...




"There is no fate but what WE make." - CEO Cyberdyne Systems Corp.
A8.30.5 Commercial
Re: Multi-threading (for A8) guru needed. [Re: EvilSOB] #384493
10/03/11 23:15
10/03/11 23:15
Joined: Jul 2001
Posts: 6,904
H
HeelX Offline
Senior Expert
HeelX  Offline
Senior Expert
H

Joined: Jul 2001
Posts: 6,904
If thread 2 can only work if thread 1 is finished, it would be make more sense to make one single new thread that does both thing sequentially. Multithreading is only useful for independent things. I would advise you to do that in a DLL that uses a comfortable Multithreading framework like OpenMP.

Just my two cents.

Re: Multi-threading (for A8) guru needed. [Re: HeelX] #384495
10/03/11 23:41
10/03/11 23:41
Joined: Apr 2007
Posts: 3,751
Canada
WretchedSid Offline
Expert
WretchedSid  Offline
Expert

Joined: Apr 2007
Posts: 3,751
Canada
Well, manipulating a terrain is only possible on the mainthread since Gamestudio doesn't impose any kind of locking mechanism on its objects. However, you can load your data and process it in a secondary thread and unlike HeelX stated this makes sense for cases where you want to load the data without blocking the main thread.
Now about synchronizing, this is done by using mutexes. A mutex (or mutual exclusion) is an object that allows only one thread to pass execution and let the other threads wait. What you want to achieve can be done by creating one mutex and one state variable. Whenever you want to read or write from/to the variable you first lock the mutex to ensure that only one thread can alter the state variable.

So, here is some pseudo code for your main thread:
Code:
void checkState()
{
    lock_mutex(globalMutex); // Lock the mutex so that only we can read it
   
    if(globalState == allowReading)
        processLoadedData(); // If the other thread signaled that we can read the data, read it.

    unlock_mutex(globalMutex); // Unlock the mutex so other threads can lock it.
}



And here is the one for your secondary thread:
Code:
void loadData()
{
    lock_mutex(globalMutex); // Lock the mutex
    globalState = waitForData; // Mark the state as "we are currently busy reading"
    unlock_mutex(globalMutex); // And then unlock the mutex so that we don't block other threads

    readDataFromDisk(); // Read the actual data from disk


    // And now set the state to ready to read.
    lock_mutex(globalMutex);
    globalState = allowReading;
    unlock_mutex(globalMutex);
}




So, now for the actual functions you need to call:
Mutexes:
Code:
HANDLE myMutex; // Handle to a mutex

myMutex = CreateMutex(NULL, FALSE, NULL); // Default security attributes, not initially locked, no name
CloseHandle(myMutex); // Destroys the mutex


WaitForSingleObject(myMutex, INFINITE); // Locks the mutex and waits infinite for it
ReleaseMutex(myMutex); // Unlocks the mutex



Threads:
Code:
HANDLE myThread; // Thread handle

myThread = CreateThread(NULL, 0, myFunction, NULL, 0, NULL); // Default security attributes, default stack size, myFunction should be the entry function, no parameter passed to the function, default flags and we aren't interested in the id of the thread (otherwise we had to pass a pointer to LPDWORD variable


DWORD myFunction(void *ignored)
{
   // Do something in the secondary thread
   return 0; // Signal succes. If your thread failed, return 1.
}



I can highly recommend you the MSDN pages for all of the above functions, they have all the details about possible parameters and links to related functions that might help you too. Last but not least, use the C FILE functions (fopen(), fread(), fclose()) to read your file as the Gamestudio file_ functions might not work in a secondary thread.


Shitlord by trade and passion. Graphics programmer at Laminar Research.
I write blog posts at feresignum.com
Re: Multi-threading (for A8) guru needed. [Re: WretchedSid] #384497
10/04/11 00:22
10/04/11 00:22
Joined: Feb 2008
Posts: 3,232
Australia
EvilSOB Offline OP
Expert
EvilSOB  Offline OP
Expert

Joined: Feb 2008
Posts: 3,232
Australia
HeelX: Blame this on my over-simplification. Thread TWO will be working on an array
of data that is SOMETIMES replaced by thread ONE.
Thread TWO will be re-reading (never writing) the array several times per update by thread ONE.

And both thread ONE and thread TWO both have many arrays to manage. So they will never (and I WANT it like this),
never be working on the same one at the same time.
And I only want one thread each for these two tasks, because they are either time-hungry(ONE) or CPU-hungry(two).
So by buffering data 'to be processed' by these threads, it lets me create a 'known' bottleneck that will allow me to
limit CPU/HDD utilisation by these tasks.


JustSid: Only just starting to read yours now...

[PS] They are actually models PRETENDING to be terrains.
And they will actually be hidden (INVISIBLE) during the vertex-adjustment phase, then swapped into replace the old one.


Last edited by EvilSOB; 10/04/11 00:31. Reason: PostScript

"There is no fate but what WE make." - CEO Cyberdyne Systems Corp.
A8.30.5 Commercial
Re: Multi-threading (for A8) guru needed. [Re: EvilSOB] #384499
10/04/11 00:40
10/04/11 00:40
Joined: Feb 2008
Posts: 3,232
Australia
EvilSOB Offline OP
Expert
EvilSOB  Offline OP
Expert

Joined: Feb 2008
Posts: 3,232
Australia
JistSid: Now Ive done a 'quick' readthrough of your post, but is using mutex's necessary? (in this situation anyway)
Read my above reply to HeelX and you'll see my more exact needs.
TO MY MIND, by using the status-flags embedded in the data-area, I am really creating a 'different' mutual-exclusion system.
I prefer mine cause the original GS thread can see the states easily too. It needs to react according to those statuses.

Note this is only from a QUICK read-through. Im now moving on to a PROPER reading.

[EDIT] I now have a better understanding, but still dont see a need for mutex's. Feel free to argue though...
Im in the midst of uilding a semi-functional example to test with. I'll post it when Im finished it, or stuck.
Question: You suggested using fopen fclose, etc. I was going to use the CreateFile API and its fellows. Any reasons against?

And thanks guys. Any input is good input. Or so my missus says. But she lies alot...


Last edited by EvilSOB; 10/04/11 06:04. Reason: edited in extra notes and a question

"There is no fate but what WE make." - CEO Cyberdyne Systems Corp.
A8.30.5 Commercial
Re: Multi-threading (for A8) guru needed. [Re: EvilSOB] #384506
10/04/11 06:04
10/04/11 06:04

M
mercuryus
Unregistered
mercuryus
Unregistered
M



Hi EvilSOB,

once apon a time (2 1/2 years ago) i posted a small and very easy example to use multithreading with lite-c.
My solution seams to be the better for your idea.

Unfortunately the example got lost from my server - i will upload the example again (in ~ 10h) - and pm you the link...

boost up your game...

Re: Multi-threading (for A8) guru needed. [Re: ] #384507
10/04/11 06:06
10/04/11 06:06
Joined: Feb 2008
Posts: 3,232
Australia
EvilSOB Offline OP
Expert
EvilSOB  Offline OP
Expert

Joined: Feb 2008
Posts: 3,232
Australia
2 and a half years !?! No wonder there wasnt much in the ubb-search function...

Thanx




"There is no fate but what WE make." - CEO Cyberdyne Systems Corp.
A8.30.5 Commercial
Re: Multi-threading (for A8) guru needed. [Re: EvilSOB] #384520
10/04/11 10:59
10/04/11 10:59
Joined: Feb 2008
Posts: 3,232
Australia
EvilSOB Offline OP
Expert
EvilSOB  Offline OP
Expert

Joined: Feb 2008
Posts: 3,232
Australia
OK guys, my brain is wearing thin, the sandman is calling, but he knows I have a shiv!

Here is the example code I said I was writing that, I feel, does everything I want.
Be warned here and now, it WILL compile and run, but the threads wont get created...
I currently have the CreateThreads remarked out, as I dont yet have a data file to test with.

But Im posting the code so if anyone can see a problem with my logic or handling OF THE THREADS just by reading
though my (rather messy but fairly well documented) code, then PLEASE point it out for me!

I'm going to try and generate a faked-up data file now IF I can dodge the sandman long enough.... He's SNEAKY!

Thanks again all...

It may look ugly here, but in SED it is much more readable
Click to reveal..
Code:
#include <acknex.h>
#include <default.c>
#include <windows.h>
//
//
//---------------------------------------------------------------------------------------
//
HANDLE Thread_ONE;		DWORD worker_ONE(void *ignored);		short*  worker_ONE_data=NULL;
HANDLE Thread_TWO;		DWORD worker_TWO(void *ignored);		short*  worker_TWO_data=NULL;
																				ENTITY* worker_TWO_entity;
long	 close_threads = 0;		//global thread terminator
//
//
//---------------------------------------------------------------------------------------
//
#define	HeightMap			"height_map.dta"
#define	HeightMapWidth		16384
#define	HeightMapHeight	12288
#define	TerrainDimensions	128
//
//---------------------------------------------------------------------------------------
//
//
void main()
{
	wait(1);					level_load(NULL);		wait(1);				diag("\n\n\n");
	draw_textmode("Times",0,32,100);				warn_level = 3;	on_esc = NULL;
	//------------------------------------------------------------------------------------
	//
	//	Thread_ONE = CreateThread(NULL, 0x00, worker_ONE, 0x00, 0x00, 0x00); 
	//	Thread_TWO = CreateThread(NULL, 0x00, worker_TWO, 0x00, 0x00, 0x00); 
	//
	//
	while(!key_esc)
	{
		wait(1);
	}
	//
	close_threads = 1;	// game over, time to kill the threads
	while(worker_ONE_data != Thread_ONE)		wait(1);	//wait until thread ONE terminated
	while(worker_TWO_data != Thread_TWO)		wait(1);	//wait until thread TWO terminated
	//
	//
	//
	//------------------------------------------------------------------------------------
	if(key_shift)	exec("notepad.exe", "acklog.txt");
	sys_exit(0);
}
//
//
//
//---------------------------------------------------------------------------------------
//
// The below file-retrieval code "largely" conjectural, as the files design not finalized
// and this code has no real error-checking and it aint optimised by a long shot...
DWORD  worker_ONE(void *ignored)		
{	//NOTE: for the record, THIS code is limited to 1.2g image files, not the API's
	var last_frame = 0;			//place to remember last frame I did processing on.
	short* data_block = NULL;	//pointer to data block supplied via worker_ONE_data
	long file=0, y, x_off,y_off, count, file_ptr_lo;	//various local workspaces
	long width=sizeof(short)*TerrainDimensions;			//various local workspaces
	Sleep(0x1000);						//snooze for a bit.  Let everything else get running.
	//
	while(!close_threads)	//loop until global terminate is triggers
	{
		if((worker_ONE_data)&&(total_frames>last_frame))
		{	
			data_block = worker_ONE_data;		data_block[1]=1;	//set status active
			x_off=(long)data_block[1];		y_off=(long)data_block[2];		//retrieve offsets
			if((x_off<0)||(y_off<0))	{	error("poo!  Illegal Offsets!");	break;	}
			SetFilePointer(file, file_ptr_lo, 0x00, 0x00);
			file_ptr_lo = x_off + (y_off * HeightMapWidth);
			file = CreateFile(HeightMap, 0x80000000, 0x00, 0x00, 0x03, 0x00, 0x00);
			if(!file)	{	error("Oh crap!  Cant find HeightMap!");	break;	}
			//start retrieving data
			for(y=0; y<TerrainDimensions; y++)
			{	
				ReadFile(file, &(data_block[y*TerrainDimensions+1]), width, &count, 0x00);
				if(count!=width)	{	error("Dammit!  Hit end-of-file!");	break;	}
				file_ptr_lo = x_off + ((y+y_off) * HeightMapWidth);
				SetFilePointer(file, file_ptr_lo, 0x00, 0x00);
			}
			CloseHandle(file);	file=NULL;		
			data_block[0] = 0;			data_block = NULL;	//set status to complete.
			worker_ONE_data  = NULL;	//disconnect from data source/s
			last_frame=total_frames;	//remember to not process more THIS frame
		}
		//
		Sleep(100);	//snooze for 100 milliseconds  (for now, will drop to 10 or 1 or zero)
	}
	worker_ONE_data = Thread_ONE;		//signal that this thread is terminated
}
//
//
//---------------------------------------------------------------------------------------
//
// The below vertex-adjustment code "largely" conjectural, as this code has only been 
// tested to COMPILE state.  MUCH testing will be needed to verify thread-safety...
DWORD  worker_TWO(void *ignored)		
{	
	var last_frame = 0;			//place to remember last frame I did processing on.
	short* data_block = NULL;	//pointer to data block supplied via worker_TWO_data
	ENTITY* entity = NULL;		//pointer to data block supplied via worker_TWO_entity
	var x, y;										//various local workspaces
	CONTACT c;		D3DVERTEX* verts; 		//various local workspaces
	Sleep(0x1000);					//snooze for a bit.  Let everything else get running.
	//
	while(!close_threads)	//loop until global terminate is triggers
	{
		if((worker_TWO_data)&&(worker_TWO_entity)&&(total_frames>last_frame))
		{	
			data_block = worker_TWO_data;		data_block[1]=1;	//set status active
			entity = worker_TWO_entity;		//retrieve pointer to affected terrain entity
			ent_getvertex(entity,c,1);			ent_setvertex(entity,c,1);		verts = c.v;
			//start modifying terrain heights
			for(y=0; y<TerrainDimensions; y++)	for(x=0; x<TerrainDimensions; x++)
				verts.y = data_block[y*TerrainDimensions+x+1];
			data_block[0] = 0;			data_block = NULL;	//set status to complete.
			worker_TWO_data = worker_TWO_entity = NULL;		//disconnect from data source/s
			last_frame=total_frames;	//remember to not process more THIS frame
		}
		//
		Sleep(100);	//snooze for 100 milliseconds  (for now, will drop to 10 or 1 or zero)
	}
	worker_TWO_data = Thread_TWO;		//signal that this thread is terminated
}




"There is no fate but what WE make." - CEO Cyberdyne Systems Corp.
A8.30.5 Commercial
Re: Multi-threading (for A8) guru needed. [Re: EvilSOB] #384535
10/04/11 16:24
10/04/11 16:24

M
mercuryus
Unregistered
mercuryus
Unregistered
M



dont know if you still need it
but

you shouldn't use any ent_ functions or in a thread!
You can store the new vectors for the vertics in a matrix (array) with one thread - and when done for all (flag?) you can call a function in your main program loop to set them to the terrain.

Re: Multi-threading (for A8) guru needed. [Re: ] #384536
10/04/11 17:15
10/04/11 17:15
Joined: Apr 2007
Posts: 3,751
Canada
WretchedSid Offline
Expert
WretchedSid  Offline
Expert

Joined: Apr 2007
Posts: 3,751
Canada
Here is another example for background loading of assets, its a bit more complete than mercuryus' and also shows the usage of mutexes (and yes, they are important as you want atomic read, since you can't gurantee that one thread alters the state while you read it (or vice versa)!)
http://cl.ly/6wN1

Btw, as for "you shouldn't use any ent_" functions: You MUST NOT! use any engine function at all in a secondary thread (you can use them on the main thread though). Like I already wrote, gamestudio doesn't impose any kind of locking for its objects so that there is no way to synchronize two or more threads.


Shitlord by trade and passion. Graphics programmer at Laminar Research.
I write blog posts at feresignum.com
Page 1 of 2 1 2

Moderated by  HeelX, Lukas, rayp, Rei_Ayanami, Superku, Tobias, TWO, VeT 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1