neural network AI

After finding a great pathfinding plug-in and figuring out that is is most likely the best discussion to create levels via Blender, I finally started to play around with AI.

I made an simple and small neural network using states of the AI as Output and own health and targets health as Input.


Basically it works like this:

(At the beginning a random state will be selected)

The AI's goal is it to exterminate! it's target.

The AI's individual states (OUTPUTS) uses points (high points happy, low points unhappy).
Certain Factors will influence (add and detract) points from the current (active) state.

After a certain break point the AI will switch to the state with the highest points (if the current state is still the highest, it will remain).

The AI's states (OUTPUTS) are:
1. stand,
2. walk towards target,
3. walk away from target,
4. strafe (left or right) ,
4. stay out of sight of the target, but approach,
5. stay out of sight of the target, and keep distance,
6. stay put,
7. run to nearest cover.

Factors (INPUT) that add or detract points are:
1. target losses health ++points,
2. AI losses heath --points,
4. being traced (aimed at) -points,
5. not able to trace target for some time (state 2 is an exception) -points (prevent stalemate ),
6. being able to trace the target itself +points.


Does anybody see any flaws here, or potential stalemates?
How would you improve on this?


Looking forward for an interesting discussion ^^


PS: And remind me to finish that cover system so I can share it.

That doesn't actually sound like a neural network but rather a state machine. Not trying to be pedantic here, but there is an incredibly huge different to the two.

In any case, it sounds like you may want to read up on the AI used in F.E.A.R. which used a somewhat similiar approach (although more modular allowing to drag and drop behaviours to create new types of enemies). Might be worth a shot to look at that, even though it's somewhat outdated these days.

, might want to let it take other enemies (/its allies) into account through flocking etc. (unless you have only very few enemies ofcourse)

Also another input I would add would be health of nearby other enemies, so if one gets hurt this enemy gets alerted and enters an alerted state.

-edit: real neural network for game AI sounds interesting but at the moment it needs huge amounts of information and doesn't seem to add much to the current tricks for game AI (while being more complicated). Maybe a hybrid form using both could be interesting but still would suck up cpu and probably hard to get it right. I think if done right in the future this could be a good AI for e.g. multiplayer fps games.

I prefer behavior trees over state machines, a slightly different theory, and imo more flexible. there are no exact standards, but e.g. the one in unreal engine 4 is quite good, it is an event driven approach. currently I'm working with it in a FPS/RTS game project and really like it. there are many examples for simple FPS enemy behaviors, and these can be implemented in lite-c too, relatively easily, I did it earlier for my RTS project. together with a good pathfinder (a navmesh is usually fine) AI creation has never been easier.

The only problem I see is too little input.

The advantage of a neural network over a state machine or others is that it can learn, adapt, change...

For it to learn what the best approach is it has to have a lot of input for just a few outputs, so it can select the best output from all the inputs and change its output if it was a wrong decision.

What I mean is this,
When you're asked to make a decision to close your bank account you're thinking about your future, your relatives, your job and your paycheck, your cash, your investments and your own entertainment, the economy's state, the bank's state, the contract for that account and so on. All this goes into making a yes/no decision to take your money right now or leave it there.

Maybe you could add some more output for cases why you need your money there or why you need your money in your pocket, but still, the input is times more than the output, because everything affects this decision...

Same goes for your game state. Where you're looking, where the enemy is looking, was he just behind a wall or he is closing on a corner where he can hide, does he have ammo near him, does he have allies near him, are those allies behind you, and so on and so on... Its a big decision!

Good luck

I have now 52 possible Inputs and 32 possible Outputs.


With that I have a little problem:

Let's say AI has the Output 1 and several inputs, let's say the inputs 1,2,3,4,5,6,7.

During those inputs the AI was able to damage it's Target and therefore received Points.

The AI will chose the Output with the highest points with the specific Input consultation, otherwise will chose an random Output.

I was wondering how I could register specific scenarios for the AI to recall.


Meaning:
Output 1, Inputs 1,2,3,4,5,6,7;
AI damaged Target with those settings, AI is happy.
Output 1 receives a point in those specific settings:
((Output 1) + (Input 1 & 2 & 3 & 4 & 5 & 6 & 7)) Point + 1;
The Output 1 however only has a point with those specific Inputs.

That Output together with those Inputs is good,
therefore in the future, if Inputs are 1,2,3,4,5,6,7, then choose Output 1, since it now has the highest points of all Outputs with that specific Input consultation, otherwise choose random Output with highest points with this input consultation.


How can I register that, for example, Output 1 receives a point, but only within those specific inputs, otherwise it should have no points.


Thanks in advance ^^

I feel like you are making it yourself too complicated with this "happiness".

The reason neural networks use feedback is to adjust the weights of nodes, but you don't really have nodes. On top of that, you give a neural network feedback while it's learning. Once the network is trained, you can just give it input and it'll produce the (hopefully) correct output based on the previous learning.
This is one of the great things about neural nets, you can teach them offline! Learning is the heavy CPU intensive process (probably GPU intensive, really). At runtime they are fast and rather resource inexpensive.

Anyway, by forcing the concept of learning and feedback into the state machine you are overcomplicating things quite a bit. Rather, you should try to define the states for your AI and what states the AI can switch to from the other states. That is to say, if your AI is in the reloading state, it can't get to the shooting state, it has to finish the reloading state. But it can start running for cover during reloading.

Next up, define how much the AI wants to reach a certain state. If it wants to see the player dead, that should be its highest goal. And then, every tick, evaluate the current state and what other states the AI can move to from where it is right now. See how much the AI wants to be in its current state (say, searching the player) versus the other states. If it can move to a state that it wants more (eg, shooting the player), then it should switch to that state.

So, let's say you have the following states and their "wanted" level:
- Searching the player: 0
- Reloading (requires gun to not be full): 1
- Shooting at the player (requires player in sight and ammo): 2

The AI starts out with an empty gun, and in the searching the player state.

Tick 1: AI checks if there are any better states and decides on the "Reloading" one because it can also satisfy its requirement. Therefore AI goes to the "Reloading" state
Tick 2: AI finishes reloading, it can no longer satisfy the requirement for the better state. Player not in sight? Okay, let's step one down to the Searching the Player state
Tick 3: Player shows up! The shooting player state can be satisfied AND it has the highest weight, therefore the AI starts shooting
Tick 4: The AI shot a couple of bullets but still has some ammo left. The shooting state is still the best option
Tick 5: Gun is empty. The AI can no longer satisfy its current state and drops to the next best state: Reloading

You get the idea. You can also have it so that not every state can be reached from every other state. That probably makes the most sense. For example, you can have it so that AI is guarding something as the least priority, while also looking out for anything suspicious which would put it in the Alarmed state, from which it can start chasing the player etc. So even though hunting the player would have a higher weight, it can go to that state unless something pings it off and it transitions to the Alarmed state.

, lots of if's (and probably some branching) that check the conditions and when the condition is met it sets some Boolean to true. You can save the Booleans in some struct which stores info/data of your AI unit, or as local variables or skills or whatever you prefer.
Than in your AI action do stuff based on which Booleans are true.
I find points less readable but using defines could make it more readable I quess.

So you are basically telling me to create a structure with over 1660 IF statements...

I was wondering if there is a way to save a specific IF combination if the combination has influenced the decision of the AI. So that there is no need to create a logical structure with over 1660 IF statements... just I hope...

Well if you really want to use that many inputs; than you could try using your points system and use if's or defines to mark output zones. Zones could e.g. look like this (just a basic example with pseudo code from the top out of my head):

etc etc. This should save save you lots of if's, and you can use defines to make it more readable ofcourse.
ps: you could remove or change the random()'s lines if you dont want the randomized results.

neural networks are not accidentally not used in games. a hierarchical state machine or a behavior tree is much simpler and faster to develop, modify/tweak, extend and to debug, and their performance is good enough.

basically they can be implemented as if-else logic, and helped by some stored parameters showing which state/branch/node is running... I did it in lite-c, it was easy after I simply hand-drawn a behavior tree.

I know, I know.

For me it is all about the ecstatics, using as few tricks is possible, an sincere experience.

Finding art in inefficiency. It makes just as much sense as playing video games ^^