Gamestudio Links
Zorro Links
Newest Posts
Why Zorro supports up to 72 cores?
by 11honza11. 04/26/24 08:55
M1 Oversampling
by 11honza11. 04/26/24 08:32
MT5 bridge not working on MT5 v. 5 build 4160
by EternallyCurious. 04/25/24 20:49
Data from CSV not parsed correctly
by EternallyCurious. 04/25/24 10:20
Trading Journey
by howardR. 04/24/24 20:04
Zorro FIX plugin - Experimental
by flink. 04/21/24 07:12
Scripts not found
by juergen_wue. 04/20/24 18:51
zorro 64bit command line support
by 7th_zorro. 04/20/24 10:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
4 registered members (VoroneTZ, Quad, EternallyCurious, 1 invisible), 828 guests, and 4 spiders.
Key: Admin, Global Mod, Mod
Newest Members
Mega_Rod, EternallyCurious, howardR, 11honza11, ccorrea
19048 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 1 of 3 1 2 3
Screen streaming #455866
11/01/15 22:51
11/01/15 22:51
Joined: Jan 2006
Posts: 968
EpsiloN Offline OP
User
EpsiloN  Offline OP
User

Joined: Jan 2006
Posts: 968
I've been searching for days, but all I can find is either too complicated for me, or it uses external software that works only as a VNC.

I want to stream my app to debian and use a raspberry as a handheld "monitor" of my laptop using sockets.

I've found (very few) examples that use D3D to access the backbuffer and write it to a file, but I have no idea if this is suitable to be streamed (not sure if it can be turned back into an image on the client) and also, I have no idea how to get the device context and stream this info using a socket (Theres only a D3D function to save the data as an image file, which is slower than sending that data directly).

I'm a newbie in D3D, but I'll get better soon.

Anything that can point me in the right direction will be appreciated laugh

Thanks.

PS.: I found this, an example by HeelX on grabbing the screen, and from what I can understand, "hBitmapdc" holds the content of the backbuffer for the desktop (lets assume I somehow point it towards the engine window grin ), how can I send it to Python socket client (what kind of data it is?) and will Python understand what it is, I mean, is it a bmp? Sorry if this sounds newbish, but I have no idea what this buffer holds grin
I found this for Python (wx) which creates an image from a buffer object (suppose a socket put that data in that object) and display it later as a streamed image in a window:
Quote:

CopyFromBuffer(self, data, format=BitmapBufferFormat_RGB, stride=-1)
Copy data from a buffer object to replace the bitmap pixel data.
Default format is plain RGB, but other formats are now supported as
well.




Here's the example from HeelX on grabbing the image:
Quote:

// Grabs the entire desktop. Make sure to set video_alpha to 0 before and to 100 after,
// to grab the desktop without the engine. Returns a BMAP* with the grabbed desktop.
//
BMAP* bmap_for_desktop ()
{
BMAP* bmap = NULL;

// get the device context of the entire desktop, including windows bar and everything
HDC hDesktopDC = GetWindowDC(HWND_DESKTOP);
if (hDesktopDC != 0)
{
// get desktop size in pixels
int desktopSizeX = GetSystemMetrics(SM_CXSCREEN);
int desktopSizeY = GetSystemMetrics(SM_CYSCREEN);

// creates a bitmap compatible with the desktop device context, in which we will later
// blit the desktop content into

HBITMAP hBitmap = CreateCompatibleBitmap(hDesktopDC, desktopSizeX, desktopSizeY);
if (hBitmap != NULL)
{
// create a memory device context compatible with the device context of the desktop. This
// context will be used to select the blit target
HDC hBitmapdc = CreateCompatibleDC(hDesktopDC);

// select the target bitmap into a the desktop-alike device context and copy the desktop
SelectObject(hBitmapdc, hBitmap);
BitBlt(hBitmapdc, 0, 0, desktopSizeX, desktopSizeY, hDesktopDC, 0, 0, SRCCOPY);

// create Gamestudio bitmap with size of the desktop
bmap = bmap_createblack(desktopSizeX, desktopSizeY, 888);
if (bmap != NULL)
{
// lock bitmap to blit from windows bitmap to Gamestudio bitmap
var format = bmap_lock(bmap, 0);
if (format > 0)
{
// we blit the bitmap pixelwise by fetching and converting the RGB components from the
// Windows bitmap and throwing it with pixel_to_bmap into the Gamestudio bitmap. This
// procedure might be faster by using GetDIBits and bmap->finalbits, but this seems to be
// safer for now

int iRow, iCol;
COLORREF colorRef;
COLOR color;
var pixel;

for (iRow = 0; iRow < bmap->height; iRow++)
{
for (iCol = 0; iCol < bmap->width; iCol++)
{
// retrieve the RGB color value as of the pixel at the specified coordinate as hexadecimal
// value 0x00bbggrr. We use the standard GDI macros to extract the component
colorRef = GetPixel(hBitmapdc, iCol, iRow);

if (colorRef != CLR_INVALID)
{
color.red = GetRValue(colorRef);
color.green = GetGValue(colorRef);
color.blue = GetBValue(colorRef);
}
// else: the pixel is outside of the current clipping region

// convert and set the retrieved color to a pixel in the format of the Gamestudio bitmap
pixel = pixel_for_vec(&color, 100, format);
pixel_to_bmap(bmap, iCol, iRow, pixel);
}
}

// blitting done, unlock Gamestudio bitmap
bmap_unlock(bmap);
}
}

// lets delete the desktop-alike device context
DeleteDC(hBitmapdc);
}

// an application must not -delete- a DC whose handle was obtained by calling the GetDC
// function. Instead, it must call the ReleaseDC function to free the DC. If such a DC is
// not freed, serious effects on painting requested by other applications can happen!
ReleaseDC(HWND_DESKTOP, hDesktopDC);
}

return(bmap);
}

Last edited by EpsiloN; 11/01/15 23:28.

Extensive Multiplayer tutorial:
http://mesetts.com/index.php?page=201
Re: Screen streaming [Re: EpsiloN] #455869
11/02/15 07:46
11/02/15 07:46
Joined: Oct 2007
Posts: 5,210
İstanbul, Turkey
Quad Online
Senior Expert
Quad  Online
Senior Expert

Joined: Oct 2007
Posts: 5,210
İstanbul, Turkey
any high performance game-streaming is done on gpu driver level and VNC is in no way suitable for game streaming.


3333333333
Re: Screen streaming [Re: Quad] #455870
11/02/15 08:37
11/02/15 08:37

M
Malice
Unregistered
Malice
Unregistered
M



<---Swimming over his head.
Click to reveal..
Could you pull the dx 9 back buffer, send it as raw , pulling it back in , then feed it to the new machines dc?
Quote:
use IDirect3DDevice9::GetBackBuffer() to obtain the back buffer surface
- use IDirect3DSurface9::GetDC() to obtain the device context.

EDIT- MORE fun with Google http://gamedev.stackexchange.com/questio...without-using-d
Yup, sorry. Hbitmap- is a Windows system struct of some type to hold a bit .. hbitmapdc- seems to be hbitmap Device Context.
Quote:
hBitmapDC: Handle to a device context of the image .......
d.
-form Microsoft.

Google was fun, but yup - have no really answers.

Last edited by Malice; 11/02/15 09:17.
Re: Screen streaming [Re: ] #455874
11/02/15 10:05
11/02/15 10:05
Joined: Jan 2006
Posts: 968
EpsiloN Offline OP
User
EpsiloN  Offline OP
User

Joined: Jan 2006
Posts: 968
That's what I'm after. Not a VNC but a custom app that'll stream my game exclusively.

I'll have to read more on device context and turning it into a regular bitmap, I guess.

I plan on using prediction for the stream, instead of compression, and I'll try to use Google's WEBP for this...So it becomes a full stream (possibly more than 30 fps) on LAN.

Thanks for the heads up Malice laugh


Extensive Multiplayer tutorial:
http://mesetts.com/index.php?page=201
Re: Screen streaming [Re: EpsiloN] #455886
11/02/15 13:27
11/02/15 13:27
Joined: Jan 2006
Posts: 968
EpsiloN Offline OP
User
EpsiloN  Offline OP
User

Joined: Jan 2006
Posts: 968
Ok, my quest continues laugh

It appears bitmap is a bitmap, there's only a difference in the order of storage as bits...(top to bottom or bottom to top).

After more reading, on Microsoft, I found out that a bitmap is represented by COLORREF objects representing 3 8-bit integers for the color and one unused 8-bit before them in hexadecimal format, that HeelX demonstrates how to disassemble into rgb values (0-255) with Get macros to a var.

I could pack that into a struct and send it through a socket to python, which could be assembled into a wxPython type and displayed on screen:
Quote:
wxBitmap is intended to be a wrapper of whatever is the native image format that is quickest/easiest to draw to a DC or to be the target of the drawing operations performed on a wxMemoryDC. By splitting the responsibilities between wxImage/wxBitmap like this then it's easier to use generic code shared by all platforms and image types for generic operations and platform specific code where performance or compatibility is needed.

Afaik wxImage is data representing a platform independent image. It contains a class RGBValue:
Quote:
class RGBValue
A simple class which stores red, green and blue values as 8 bit unsigned integers in the range of 0-255.

So, I now only have to test it somehow, without having my raspbian booted up grin and convert somehow the data into an wxImage to be displayed by a DC.

If anyone has any ideas, feel free to express yourself grin


Extensive Multiplayer tutorial:
http://mesetts.com/index.php?page=201
Re: Screen streaming [Re: EpsiloN] #455895
11/02/15 15:32
11/02/15 15:32
Joined: Apr 2007
Posts: 3,751
Canada
WretchedSid Offline
Expert
WretchedSid  Offline
Expert

Joined: Apr 2007
Posts: 3,751
Canada
I hate to be a buzzkill, but this task is way more complicated than you might think it is. For starters, you don't want to actually bitmaps, as they are large and will clog up your network very very fast. Instead, you probably want to live encode it into a video and stream that, as those provide adaptive quality, if needed, but can also cope with package loss much much better. For a quick and dirty start, anything keyframe based will probably work, although h.264 is probably the best since there is plenty of documentation and code for it around.

That is not to say that you still have to grab your screen, because you have to, but you will probably want to dump the result directly into something like ffmpeg to get the video encoding going, and then send the resulting frame via a socket and then assemble the video back together on the other end. And don't even bother with converting the backbuffer to a BMAP or anything, you want raw access and then do as little as possible with it to avoid the penalty of converting the result around. So ideally your backbuffer is already in a format that is directly supported by ffmpeg and preferably you have a way to get it asynchronously without blocking the rendering (so probably triple buffer everything or so).

The receiving side on the other hand is much simpler, although I'm not sure if the Raspberry Pi has enough power to deal with the video decoding. Probably not, but I think there you can get some hardware decoding action if you pay for a license. But yeah, pretty much you just wait for a keyframe to come along and then start assembling the video frame by frame.


Shitlord by trade and passion. Graphics programmer at Laminar Research.
I write blog posts at feresignum.com
Re: Screen streaming [Re: WretchedSid] #455901
11/02/15 18:23
11/02/15 18:23
Joined: Jan 2006
Posts: 968
EpsiloN Offline OP
User
EpsiloN  Offline OP
User

Joined: Jan 2006
Posts: 968
I too gave that a thought and tried to do some calculations.

If I use the raw data in the buffer, it'll be around 1mb/f so 60mb/s in 800x420 24bit.

But, I intend to compress it somehow, although I never thought about video encoding.
I don't want to convert the raw data of the backbuffer, because this will slow the process, I want to just compress it and send it to be decompressed in raw data and then the raspberry will have to use it as is (device independent...)
That way, the raspberry will only need to decompress, not also convert or resize or anything...

I was thinking something like jpeg and then using the WEBP format, or directly using the WEBP (I didn't see any working examples, but Google did explain how the process works, so I thought I could replicate it in the future, after I get a few frames going.)

I'm not sure if the raspberry could handle any kind of decompression if its 60fps, but I'll experiment if it fails.

I'll give a look at h.264 later today laugh I always download that kind of movies, but never thought about using this grin hah, irony.


Extensive Multiplayer tutorial:
http://mesetts.com/index.php?page=201
Re: Screen streaming [Re: EpsiloN] #455903
11/02/15 18:32
11/02/15 18:32
Joined: Apr 2007
Posts: 3,751
Canada
WretchedSid Offline
Expert
WretchedSid  Offline
Expert

Joined: Apr 2007
Posts: 3,751
Canada
You still need to encode and decode the jpeg, and that is quite costly. A jpeg really doesn't give you anything over a raw buffer in terms of how fast you can push it to the GPU to get it rendered, or how fast you can get it from the GPU to the socket, because in both cases you have to transform the image.

Really, use h.264 or any other video format which uses keyframes, as that will drastically improve quality over jpeg and decrease bandwidth need. Plus, your jpeg is not immune to packet loss, whereas h.264 is (although with reduced quality as a result).


Shitlord by trade and passion. Graphics programmer at Laminar Research.
I write blog posts at feresignum.com
Re: Screen streaming [Re: WretchedSid] #455905
11/02/15 18:43
11/02/15 18:43
Joined: Jan 2006
Posts: 968
EpsiloN Offline OP
User
EpsiloN  Offline OP
User

Joined: Jan 2006
Posts: 968
Don't worry, you've convinced me to use a video encoding laugh I don't have a clue how it works, but I know it'll work well because its a stream and pixel perfection isn't needed, image formats are better suited for a still image, right?

Ok, but what do you mean I don't want bitmaps, exactly? Sorry, but I'm new to the D3D stuff laugh
I can still safely use GetPixel(hdc,col,row) to access the raw data and pass it pixel by pixel to be encoded, or is there some sort of array that holds all this data?

PS.: And, another quick question, haven't asked google yet, but how do I get the device context that the Acknex uses? Didn't find any pointers in the manual... Sorry if this is a stupid question.


Extensive Multiplayer tutorial:
http://mesetts.com/index.php?page=201
Re: Screen streaming [Re: EpsiloN] #455926
11/03/15 10:56
11/03/15 10:56
Joined: Apr 2007
Posts: 3,751
Canada
WretchedSid Offline
Expert
WretchedSid  Offline
Expert

Joined: Apr 2007
Posts: 3,751
Canada
Originally Posted By: EpsiloN
I know it'll work well because its a stream and pixel perfection isn't needed, image formats are better suited for a still image, right?

That's right. However, one important thing is that for example jpeg is not pixel perfect but instead applies destructive compression on the image, ie you end up artifacts in the image that you can't get out.


Originally Posted By: EpsiloN
Ok, but what do you mean I don't want bitmaps, exactly? Sorry, but I'm new to the D3D stuff laugh
I can still safely use GetPixel(hdc,col,row) to access the raw data and pass it pixel by pixel to be encoded, or is there some sort of array that holds all this data?

Pixel by pixel is way too slow, you want to get a raw buffer of the data, preferably in a format that is already supported by ffmpeg or whatever encoding library you end up using. The less operations there are between grabbing the data and getting it to the encoder the better, so ideally you want to do no work at all here and just copy the data in a suitable buffer (or even better, get a suitable buffer provided by whatever!) and then send that off to the encoder.

Originally Posted By: EpsiloN
PS.: And, another quick question, haven't asked google yet, but how do I get the device context that the Acknex uses? Didn't find any pointers in the manual... Sorry if this is a stupid question.

No freaking idea. Not a stupid question at all, but I'm not the right one to answer it. Someone else will probably be able to help you out there.


Shitlord by trade and passion. Graphics programmer at Laminar Research.
I write blog posts at feresignum.com
Page 1 of 3 1 2 3

Moderated by  HeelX, Lukas, rayp, Rei_Ayanami, Superku, Tobias, TWO, VeT 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1