Hello everyone,

I am currently working on finalization of my project. I am supported by Kartoffel, who wrote the rendering pipeline and some of the tools for the game. There's a serious crash bug in the game that we just can't get by. The project has become relatively complex and the error occurs very randomly, and we can't find the cause even after literally several hundred hours of work. I hope that someone from the forum or the developers can help us out, or at least give us an advice. I'd also like to make sure that this is not a global engine bug - so in case you experience similar issues, please let us know. Things are extremely urgent right now for organizational reasons and because I'm planning to launch soon, so I'd be more than grateful for any help!

First off, some basic information: I am developing the game on a Windows 7 system, but have been testing regularly on Win 10 systems. For the last few months however, I did not run these tests as regularly as before, since I've completely focused on programming user interface, menu, DLL implementation (Steam and misc file operations). It's possible that the crash bug has come in somewhere during that time, but I can't say for sure. It may have also been there for a much longer period, and I just didn't notice before due to the lower complexity of the game at that time.

Bug Description:

• The project produces random crashes under Windows 10 (so far tested on five systems).
• The game runs fine under Windows 7.
• Both the published and source versions are affected.
• The crash can happen at any time: Sometimes the whole game can be played through, sometimes it already crashes after a few seconds.
• The cause of the crashes is very random and difficult to pinpoint. These are our best practices in terms of reproduction - the problem appears:
  - while attempting to load levels (no matter if triggered by gameplay functions or by user input/UI)
  - when pausing the game (= setting freeze_mode to 2 + calling some UI functions)
  - when moving the mousecursor over or away from a specific UI element (without clicking)
  - less often during or after a message is displayed, or at any random point in the game.

The crash usually appears in one of these forms:

• 1) Often:
  The screen freezes for a second
  A standard Windows error message appears:"acknex.exe stopped working"
• 2) Less frequent:
  An Acknex error message appears: "Script crash in load_map"
  After closing the message it continues as described in [1] (in most of the cases)
  Please note: This behavior is not limited to a certain level. It also happens with levels that work fine in any other case.
• 3) Rarely:
  During the loading process an Acknex error message appears: "Bad file format: stage####. wmb"
  After closing the message it continues as described in [1]
  Please note: This also happens with levels that otherwise work without any problems.

Here's our current assumption, as we already expressed in a different thread:


Measures taken so far:
As I said, the bug is difficult to narrow down. We have already taken numerous measures for error analysis - unfortunately without noticeable improvement. If there was an improvement, it was in the form that the occurrence of the crash was delayed, or happened under different circumstances. Our latest approaches include:

A) Disabling plug-ins
We do use some DLL plugins, mostly self-written ones. Of course, we have already tried to deactivate these plugins completely and also made sure to remove them from the acknex_plugins folder. It didn't do any good.

B) Narrowing it down
By turning off large blocks of code and/or complete features (menu, UI, AI, sound, player, controls + more + combinations), we have tried getting to the root of all problems. Wasn't too successful either. If there was a perceptible improvement, we re-activated line by line to see were it came from, but usually to no avail. During one of these tests I noticed the following erratic behaviour:

• I've put a 'return' right at the beginning of the following function. I did not experience the same crashing behaviour as before.
  
  Code:

  function player_spawn()
  {
  	return;
  	(...)
  }
  
  

• While re-adding lines of code, I noticed that it didn't matter what I put before the 'return'. Anything, even a comment, would re-enable the crashes.
  
  Code:

  function player_spawn ()
  {
  	// test
  	return;
  	(...)
  }
  
  

It is possible that this was just a coincidence. But I thought it was strange enough to mention it here.

C) Disabling certain model files
Our main menu uses eight background models spawned via ent_create and animated by code. While narrowing things down, I also tried to disable these models, just to see what would happen. The crash actually occurred less frequently afterwards. It did happen anyway, only much later and less frequent. I wonder where this came from. Are these model files corrupted? Could there be a mismanagement of internal memory areas when I load these models? When I don't load them, could there be kind of a "shift" in memory allocation, so that the crash occurs less often? I'm not sure what this behaviour tells me.

D) Replacement all model files
Encouraged by the partial success described before, and due to the fact that some users on this forum reported bugs with "corrupt" model files, I replaced all 1,615 models in the project folders with a simple dummy object. It didn't fix the crash, but I don't think it occurred that frequently anymore. Of course, it was difficult to really "play" the game this way, so I did not carry out extensive tests. Is it possible that other file types could be corrupted? Should I try to replace sounds, media and textures with dummies?

E) Manual termination of all "critical" functions
A very, very time-consuming approach that I have been pursuing over the last few days: I make sure that all "critical" functions are terminated before loading. In detail: I track all potentially critical loops by having them count up a variable at the beginning of the function. The level loading routine waits until all these functions are finished and the sum of all variables reaches 0. It didn't improve anything (but at least I was able to fix some sporadic pointer errors this way).

F) Data analysis
Of course we have studied the acklog, built in countless diag-instructions, use our console and write out a tasklist in situations we think are critical. Problematic variables and objects are monitored via on-screen debug displays. Only thing we haven't tried yet is to implement a ton of sys_marker instructions, as Superku recommended. I'm not sure if this could help us any better pinpointing the cause of the problem, but it might be worth a try.

Next steps:
Here's what I could still try. Don't have much more on my list right now, I must say.

I) Check/disable windows.h functions
We use some features from windows.h - could there be a problem with deprecated instructions that could lead to Win10-only crashes?

II) Check plugin source code
Although I already ruled out the plugins as a cause, I'm going to go through them step by step and look for possible error sources.

III) PROC_GLOBAL
In the course of the menu programming during the last few months, a quite large number of PROC_GLOBAL-Instructions have found the way into my code. Since Superku mentioned that this function may not always work correctly, I will soon deactivate it for testing purposes.

IV) Further narrowing it down
We still have a few ideas about which parts of the code we could investigate further, even though I have the feeling this won't shed any more light on the case. We'll try this during the next days.

Related threads:
http://www.opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=470982#Post470982


TL;DR: Game crashes only under Win10, tried almost everything, please help!