Why Programming a Campaign Game is Very Hard

[LoBaron]

ORIGINAL: fcharton
Thinking of it, this is pretty much what we'd like the AI to do in the game : it should be able to handle all the complexity in the game, yet, when assessing a situation, not get lost in details...

This is actually pretty much where I am struggling when playing WitP AE... [:D]

[LoBaron]

ORIGINAL: fcharton
This is pretty much the idea, but it is not specific to the human brain, and I don't think it is a product of evolution. It is a very general problem. We handle new situations by relating them to similar ones we have already seen and handled. Now, complex situations depend upon a lot of parameters, which never are quite the same twice. The more complex, the less similar, and the more difficult it becomes.

If we define intelligence as the ability to handle complex situations, it means there has to be a mechanism which allow us to simplify those situations, and find the "right similarities" that will allow us to solve the problem at hand. I believe that understanding how this "reduction" happens is the key to artificial intelligence and machine learning.

Is there a distinction between a complex situation and a complex task seen from this perspective?

A game like WitP AE provides a very vague definition of the term "to win" (VP´s left aside which could cause
their own problems when providing it as a condition exclusively).
I think every player has his own understanding of what winning means on a global scale, and this is
not neccesarily a sum of all victories on a tactical or local level. To me it looks like the terms "complex situation"
(interpreting available data for a present date, extrapolating the date to predict future events,..)
and "complex task" (as "winning WitP") are dissimilar.

This was the reason in the first place why I deviated to evolution. The task "provided" by evolution is pretty
basic: survive and reproduce, quite similar to a computer virus.
This situation provides a simple task which acts as a overall governor for behaviour in a complex enviroment.

To me it seems like in the current AI developement, and also when focusing on WitP this is not so.

[mike scholl 1]

ORIGINAL: herwin
This is better understood. In mammals, experience is replayed during sleep. A x20 compression in time produces the 'sharp waves', so every meaningful experience is replayed over and over. Thereafter each access to an experience in trace memory rewrites it in the direction of 'making sense'. Still, odour memories can be learned on one presentation. The activation presented to the learning system consists of the sensory afference, the motor efference, intermediate cortical and hippocampal activations, an evaluation of the unexpectedness of the experience, all emotional responses to the experience, AND the payoff associated with the experience. So there is a teaching signal. Learning seems to be Hebbian, not via backpropagation--pace Paul Werbos.

Harry. It seems to me that modeling all the functions of the human brain is an incredibly difficult task. Why don't you suggest to your friends engaged in this field that they take it in "stages". Obviously, the decision making perameters of WITP-AE ( complex though they may be ) are child's play compared to those of the human brain dealing with all the vaguarities of life itself. So why don't you do all of us a favor and propose that they solve "AE" as a first step? We'd get an AI that actually works, they'd get a feeling of accomplishment and some practice, and everyone could be happy.

Seems like a good idea to me... [:D][:D][:D]

[LoBaron]

OTOH, as long as the New And Improved AI does not show grief and despair in case
of defeat, I prefer PBEM... [:D]

[herwin]

ORIGINAL: mike scholl 1

ORIGINAL: herwin
This is better understood. In mammals, experience is replayed during sleep. A x20 compression in time produces the 'sharp waves', so every meaningful experience is replayed over and over. Thereafter each access to an experience in trace memory rewrites it in the direction of 'making sense'. Still, odour memories can be learned on one presentation. The activation presented to the learning system consists of the sensory afference, the motor efference, intermediate cortical and hippocampal activations, an evaluation of the unexpectedness of the experience, all emotional responses to the experience, AND the payoff associated with the experience. So there is a teaching signal. Learning seems to be Hebbian, not via backpropagation--pace Paul Werbos.

Harry. It seems to me that modeling all the functions of the human brain is an incredibly difficult task. Why don't you suggest to your friends engaged in this field that they take it in "stages". Obviously, the decision making perameters of WITP-AE ( complex though they may be ) are child's play compared to those of the human brain dealing with all the vaguarities of life itself. So why don't you do all of us a favor and propose that they solve "AE" as a first step? We'd get an AI that actually works, they'd get a feeling of accomplishment and some practice, and everyone could be happy.

Seems like a good idea to me... [:D][:D][:D]

It's less hard than it sounds, because the modules are rather simple in what they do. It's the learning process I need to figure out.

And, yes, an AI for AE is probably a good first step.

[herwin]

Reference: Girshick, Landy & Simoncelli (2011) Cardinal rules: visual orientation perception reflects knowledge of environmental statistics, Nature Neuroscience, v. 14, Nr. 7, 926-939, July 2011.

Visual orientation follows Bayes Rule. In other words, what we perceive is the result of applying Bayes Rule to combine expectation and sensory input. And future expectation uses that combination.

So if you want a biological AI, that's what you do.

[Buck Beach]

ORIGINAL: zzodr

One thing i've learnt from computer games: Computers cheat.
But I can always reload my save game. [;)]

An AI player after my own heart. Do overs are an intricate part of my grand strategy. My tired old (alcohol damaged) brain can not keep up with all the things I should do in any given turn.

Buck

Oops, sorry to have fouled this serious thread with my antidotes.

[herwin]

This post will probably go straight over the head of most readers, but there are a few who might find it leads to new ideas.

Biological learning is Hebbian learning. That is the strength of the connection from one neuron to another is increased if the destination neuron is supposed to spike in response to an input. That can take place in one of two ways:

1. A destination neuron reliably spikes after received the input to be learned. Previous input has depolarised the neuron, knocking magnesium ions out of NMDA channels and allowing calcium to enter via those channels. That triggers a weight increase for the synapse carrying the input. Hence the neuron learns the causal association between the two inputs and spikes more and more reliably. Learning rate scales with reliability here.

2. Alternatively, the cell receives a dopaminergic 'this is important signal' before the input to be associated with the desired output. (Dopamine serves as a reward signal and GABA serves as a negative reward signal in mammals.) This works similarly. In this case, learning rate scales with the unexpectedness of the reward.

The pattern learned is the result of a Bayesian update of previous expectation (the prior) based on the current stimulus pattern. It's learned as a package, including reward signals and actions. Neurons seem to converge to Bayesian estimators with learning.

So a BI for a strategic game would learn causal patterns. These patterns include the expected reward of the pattern, allowing the BI to select those actions (which are part of the pattern) that produce the highest reward. This is different from machine learning.

Time to sleep on it to help me nail down the details.

[LoBaron]

Tbh, herwin, this was one of the easier to read posts from your side. [;)]

Thanks for the detailed information, I had only a rough picture about Hebbian learning.

What I wonder is whether a game like WitP is optimal for integration of a Hebbian AI,
(comparing real time to turn based planning).

In case it interests you, I wonder if you should take a look at Distant Worlds, also
published by matrix games, as from how I understand it it seems more suitable for experiments
on that level.

[herwin]

ORIGINAL: LoBaron

Tbh, herwin, this was one of the easier to read posts from your side. [;)]

Thanks for the detailed information, I had only a rough picture about Hebbian learning.

What I wonder is whether a game like WitP is optimal for integration of a Hebbian AI,
(comparing real time to turn based planning).

In case it interests you, I wonder if you should take a look at Distant Worlds, also
published by matrix games, as from how I understand it it seems more suitable for experiments
on that level.

Neurons are not a good match to the requirements on a game AI. Cell assemblies are a better match, because their activation can be treated as a collection of probabilities associated with hexes on the game board. If I am just concerned about is *where* each LCU/ship/squadron is, I can describe that as a probability weighting on an collection of hexes. I can even do this by listing a few hexes with substantially larger probabilities than the default, and associate the default probability with any other hexes the unit may enter. I may also address the target of the unit in the same way. What happens each turn then updates that data structure.

A plan for a unit is a pattern of shifting probabilities for that unit over a number of future turns.