Whatever AI you try to make it will still need some kind of training, just like every living being does. You know that something hurts because it really does hurt, which is a feeling you want to avoid. At some point you may know what food you like a lot and which you don't because you like the taste or you hate it, which is also something you kinda just know, although this may change over time probably based on lots of different influences, which in theory could be simulated.

Everybody learned by trying and getting some kind of feedback and others sharing their experience. Also at some point it is possible to make connections like "thing x hurts and thing y looks similar so it probably hurts too".

I really don't think a neural net is far off from what you are talking about and might even be capable of that given a complex simulation of human life, which however won't be easy to create and teach...