Why Programming a Campaign Game is Very Hard

[Shark7]

ORIGINAL: LoBaron

ORIGINAL: Shark7
Actually a better way of seeing it is that while a computer is capable of coming to a decision based on a number of variables, it is not capable of coming to variable conclusions.

That is the strength that the human mind has over an AI. An AI is only capable of coming to the conclusions it is programmed to come to. Take 100 computers with the same code and same variables, and they will come to the same comclusion...even with the addition of a randomizer, they will still fall into a predictable range. Take 100 humans and present them with a problem, and you will likely come to 100 different methods of arriving at a solution.

Add to that the human mind is capable of 'thinking outside the box'. An AI is very limited to the box it has defined in its code.

AIs may be able to crunch a bunch of numbers and do it a lot faster than you or I, but you and I have the advantage of creativity. [;)]

100 Humans presented with a problem may yield 100 differet methods of arriving at a solution. But theres a simple underlying
reason:
Those humans have 100 different brains, none of them operates the same on a biochemical level, and additionaly they have
100 different sets of, as fcharton said, a posteriori knowledge gathered through different experiences made in their
lifetime.

Comparing this to 100 identical AIs who access an identical dataset is comparing apples to oranges.
A comparision on that level only is valid as long as you compare the capabilities and differences
of one human brain to a single AI. And there the above conclusions would not be possible.

Edit: the a posteriori approach is very intriguing, just because of what was written above. It would
obviousely result in different behaviour of 2 AIs, even if they are identical in their base structure
because they access different datasets.

The complex problem is how to bring an AI to identify data needed to adress a specific task if the dataset
is not present from the beginning. So how do you enable an AI to both, create and categorize new datasets,
and successfully use the available data in a way it makes sense?

Even with single AI and single Brain, the difference is noticable. The human brain reasons through a completely different method...because the human is capable of thinking of the future, not just the present set of data.

It's like a game of Chess...sure a computer that is programmed with all the rules and moves of chess can analyze the humans moves and counter. However, a chess champion doesn't just see this move, he is thinking 10 moves ahead and how an opponents moves might affect later moves.

It's hard to explain...but perhaps the best description is that the AI thinks in the here and now, while a human weighs the cause and effect?

[herwin]

ORIGINAL: LoBaron

ORIGINAL: Shark7
Actually a better way of seeing it is that while a computer is capable of coming to a decision based on a number of variables, it is not capable of coming to variable conclusions.

That is the strength that the human mind has over an AI. An AI is only capable of coming to the conclusions it is programmed to come to. Take 100 computers with the same code and same variables, and they will come to the same comclusion...even with the addition of a randomizer, they will still fall into a predictable range. Take 100 humans and present them with a problem, and you will likely come to 100 different methods of arriving at a solution.

Add to that the human mind is capable of 'thinking outside the box'. An AI is very limited to the box it has defined in its code.

AIs may be able to crunch a bunch of numbers and do it a lot faster than you or I, but you and I have the advantage of creativity. [;)]

100 Humans presented with a problem may yield 100 differet methods of arriving at a solution. But theres a simple underlying
reason:
Those humans have 100 different brains, none of them operates the same on a biochemical level, and additionaly they have
100 different sets of, as fcharton said, a posteriori knowledge gathered through different experiences made in their
lifetime.

Comparing this to 100 identical AIs who access an identical dataset is comparing apples to oranges.
A comparision on that level only is valid as long as you compare the capabilities and differences
of one human brain to a single AI. And there the above conclusions would not be possible.

Edit: the a posteriori approach is very intriguing, just because of what was written above. It would
obviousely result in different behaviour of 2 AIs, even if they are identical in their base structure
because they access different datasets.

The complex problem is how to bring an AI to identify data needed to adress a specific task if the dataset
is not present from the beginning. So how do you enable an AI to both, create and categorize new datasets,
and successfully use the available data in a way it makes sense?

That's why I'm attracted to Gibbs Sampling. Not only are parameters estimated, 'meta-parameters' describing the distribution of the parameters are estimated. It's a learning algorithm, and uses random numbers to help make sure more of the space of possibilities is explored. The other thing about it that I like is it's implicit, rather than explicit. This means, it allows the data to tell its story and doesn't impose a story on the data.

[herwin]

ORIGINAL: Shark7

ORIGINAL: LoBaron

ORIGINAL: Shark7
Actually a better way of seeing it is that while a computer is capable of coming to a decision based on a number of variables, it is not capable of coming to variable conclusions.

That is the strength that the human mind has over an AI. An AI is only capable of coming to the conclusions it is programmed to come to. Take 100 computers with the same code and same variables, and they will come to the same comclusion...even with the addition of a randomizer, they will still fall into a predictable range. Take 100 humans and present them with a problem, and you will likely come to 100 different methods of arriving at a solution.

Add to that the human mind is capable of 'thinking outside the box'. An AI is very limited to the box it has defined in its code.

AIs may be able to crunch a bunch of numbers and do it a lot faster than you or I, but you and I have the advantage of creativity. [;)]

100 Humans presented with a problem may yield 100 differet methods of arriving at a solution. But theres a simple underlying
reason:
Those humans have 100 different brains, none of them operates the same on a biochemical level, and additionaly they have
100 different sets of, as fcharton said, a posteriori knowledge gathered through different experiences made in their
lifetime.

Comparing this to 100 identical AIs who access an identical dataset is comparing apples to oranges.
A comparision on that level only is valid as long as you compare the capabilities and differences
of one human brain to a single AI. And there the above conclusions would not be possible.

Edit: the a posteriori approach is very intriguing, just because of what was written above. It would
obviousely result in different behaviour of 2 AIs, even if they are identical in their base structure
because they access different datasets.

The complex problem is how to bring an AI to identify data needed to adress a specific task if the dataset
is not present from the beginning. So how do you enable an AI to both, create and categorize new datasets,
and successfully use the available data in a way it makes sense?

Even with single AI and single Brain, the difference is noticable. The human brain reasons through a completely different method...because the human is capable of thinking of the future, not just the present set of data.

It's like a game of Chess...sure a computer that is programmed with all the rules and moves of chess can analyze the humans moves and counter. However, a chess champion doesn't just see this move, he is thinking 10 moves ahead and how an opponents moves might affect later moves.

It's hard to explain...but perhaps the best description is that the AI thinks in the here and now, while a human weighs the cause and effect?

That's right--programs don't do cause and effect until you program it in. Our brain seems to do cause and effect from birth--it's how we learn.

Interestingly, last month they identified neurons in the hippocampus that time things and can be used as delay timers. So we're preadapted for planning.

[mdiehl]

I'd like my agents to emerge from the training and not have to be programmed in.

If you're talking about making operational plans, that's a heck of a task set before you. I think programming it in probably works well on the ship/section level since the "rules" are pretty well documented in the various navies and air forces' tactical doctrine. I was thinking for ex of the agent based models of flocking behavior. Probably something like that would work quite well at the section or DDdiv or CAdiv level. Well, I guess I should not say "probably" because we can't really know until it's been tried.

Your AI you're talking about is basically something that learns its opponents habits. If you want it to learn approximating human learning at this sort of thing (which is done by lots of people acting in concert analyzing data and making operational plans), it'd have to do something like this.

Set up and implement several operational plans.
Choose one and submit that as a player turn.
Run all the others as separate player turns to assess the range of credible outcomes.
Compare the results from the turn that it actually submitted to these other outcomes.
Assess the utility of the results.
Assess the differences between the best utility outcome and the actual outcome in order to prepare for future planning.

Kind of like having a bunch of AIs working at once, but only one of them gets to make the decision each turn.

There have to be three or four different ways you could work that too, rather than doing it the way I outlined. For ex, you could have a "passive AI" a "deceptive AI" an "aggressive AI" and have them each run turns, after several turns the one whose projected or actual output is best gets to call the shots for a while. If its performance falls off comopared to the others, one of the other AIs takes over for a while.

In that way your AI could react to varying opfor operational plans and other general changes to the map situation etc.

[herwin]

ORIGINAL: mdiehl

I'd like my agents to emerge from the training and not have to be programmed in.

If you're talking about making operational plans, that's a heck of a task set before you. I think programming it in probably works well on the ship/section level since the "rules" are pretty well documented in the various navies and air forces' tactical doctrine. I was thinking for ex of the agent based models of flocking behavior. Probably something like that would work quite well at the section or DDdiv or CAdiv level. Well, I guess I should not say "probably" because we can't really know until it's been tried.

Your AI you're talking about is basically something that learns its opponents habits. If you want it to learn approximating human learning at this sort of thing (which is done by lots of people acting in concert analyzing data and making operational plans), it'd have to do something like this.

Set up and implement several operational plans.
Choose one and submit that as a player turn.
Run all the others as separate player turns to assess the range of credible outcomes.
Compare the results from the turn that it actually submitted to these other outcomes.
Assess the utility of the results.
Assess the differences between the best utility outcome and the actual outcome in order to prepare for future planning.

Kind of like having a bunch of AIs working at once, but only one of them gets to make the decision each turn.

There have to be three or four different ways you could work that too, rather than doing it the way I outlined. For ex, you could have a "passive AI" a "deceptive AI" an "aggressive AI" and have them each run turns, after several turns the one whose projected or actual output is best gets to call the shots for a while. If its performance falls off comopared to the others, one of the other AIs takes over for a while.

In that way your AI could react to varying opfor operational plans and other general changes to the map situation etc.

Yes, in general. That is how intelligence seems to work. An action-generation system suggests a collection of possibilities, and the conscious brain allows one to be acted upon.

[fcharton]

Hi LoBaron,

ORIGINAL: LoBaron
The complex problem is how to bring an AI to identify data needed to adress a specific task if the dataset
is not present from the beginning. So how do you enable an AI to both, create and categorize new datasets,
and successfully use the available data in a way it makes sense?

I believe an AI can do this, once it is given a metric that measures the similarity between datasets. With such a distance, you can build categories from scratch by clustering available data, and create new datasets by "averaging" similar ones.

In other words, the AI processes new datasets by combining old ones. This is very close to the way we learn by experience. We never see the same situation twice, but we can spot similarities between new problems and past ones, and react accordingly.

The difficulty, then, is to define a metric to classify events (and refine it using available data). At this point, there is a bad trade off, sometimes known as Bellman's curse.

- to correctly classify complex datasets, you need to take a lot of parameters into account,
- this means your distance (and your datasets) will "live" in a high dimension phase space
- but... the higher the dimension, the further apart two average data points,
- which makes it difficult to find similarities... and therefore learn...

Finding ways to build efficient algorithms in high dimensions is pretty much the subject of modern research on machine learning.

@Harry,
I suppose you are familiar with Rosenblatt's Perceptron and the backprop neural nets that were oh so fashionable in the 80s. You might be interested in a book by V. Vapnik, The Nature of Statistical Learning Theory ( http://www.amazon.com/Statistical-Learn ... 0387987800 ) which offers a lot of insight about how and why neural nets (and similar connectionnist models) work. I suspect a lot of Vapnik's remarks apply to your problem.

It is also an interesting attempt at trying to define machine learning (and machine intelligence), and a good introduction to modern methods, like Support Vector Machines, kernel based methods, and Hilbert Schmidt norms.

Francois

[herwin]

ORIGINAL: fcharton

@Harry,
I suppose you are familiar with Rosenblatt's Perceptron and the backprop neural nets that were oh so fashionable in the 80s. You might be interested in a book by V. Vapnik, The Nature of Statistical Learning Theory ( http://www.amazon.com/Statistical-Learn ... 0387987800 ) which offers a lot of insight about how and why neural nets (and similar connectionnist models) work. I suspect a lot of Vapnik's remarks apply to your problem.

It is also an interesting attempt at trying to define machine learning (and machine intelligence), and a good introduction to modern methods, like Support Vector Machines, kernel based methods, and Hilbert Schmidt norms.

Francois

Oh, yes, I know that work and many of those folks. Vapnik's book is a Springer publication, hence expensive even on Kindle, but it looks very useful. I certainly can't get it at the local uni library.

[JWE]

ORIGINAL: fcharton
The difficulty, then, is to define a metric to classify events (and refine it using available data). At this point, there is a bad trade off, sometimes known as Bellman's curse.

- to correctly classify complex datasets, you need to take a lot of parameters into account,
- this means your distance (and your datasets) will "live" in a high dimension phase space
- but... the higher the dimension, the further apart two average data points,
- which makes it difficult to find similarities... and therefore learn...

Finding ways to build efficient algorithms in high dimensions is pretty much the subject of modern research on machine learning.

Hi Francois; That's pretty much where LTI is, as you may have guessed from the materials I sent you. We started very simple, with only an 18 element parametric space, but as soon as we expanded them upwards, convergence was problematic, to say the least. There seems to be an inherent instability in an open model. It's almost as though learning must be constrained to a rather truncated set of conditions, with proportional limitations. And that's yet another layer. Woof!!!

[ed] But this is such a totally bizarre place to be discussing things like this. It makes me wonder. If there's that much horsepower, why ain't they at the 'Tute AI site? Or Cal Tech's? Or Yale's, or Harvard's, or Chicago's, UCI's. Michigan's, or (damn me for saying it) USC's? There's a lot out there.

[herwin]

ORIGINAL: JWE

ORIGINAL: fcharton
The difficulty, then, is to define a metric to classify events (and refine it using available data). At this point, there is a bad trade off, sometimes known as Bellman's curse.

- to correctly classify complex datasets, you need to take a lot of parameters into account,
- this means your distance (and your datasets) will "live" in a high dimension phase space
- but... the higher the dimension, the further apart two average data points,
- which makes it difficult to find similarities... and therefore learn...

Finding ways to build efficient algorithms in high dimensions is pretty much the subject of modern research on machine learning.

Hi Francois; That's pretty much where LTI is, as you may have guessed from the materials I sent you. We started very simple, with only an 18 element parametric space, but as soon as we expanded them upwards, convergence was problematic, to say the least. There seems to be an inherent instability in an open model. It's almost as though learning must be constrained to a rather truncated set of conditions, with proportional limitations. And that's yet another layer. Woof!!!

How does it happen so fast in real brains? Yes, the experience can be replayed at 20x real-time during sleep, but we see single-presentation learning of odors in rats and rabbits without sleep. We also see the formation of complex concept assemblies in human brains in a very few days. Learning in vivo is very fast.

[fcharton]

Hi John,

You've got to get your statboys to read Vapnik... [;)] If they get nosy, just assign them the one published by Wiley (Statistical Learning Theory), real tough stuff...

There are two ways around the problem of dimensionality.

The traditional (Soviet) one is to add artificial constraints to your problem. That's pretty much what regularisation is all about. Say, you have a huge number of parameters, but you add a lot of implicit constraints (typically small norms in parameter space, or stuff like this), which act as a way to force convergence, by selecting one solution among many feasible ones. Basically, you add small "constraint penalties" to the function you minimize, and this stabilizes the calculation.

The modern approach tries to attack the dual problem (a typical aspect of modern maths). Instead of minimising the distance between model and data under constraints, you minimize the constraints in a very large feature space (billions of parameters sometimes happen in practice), under the requiement that distance to data be zero, or below a very small value. In other word, you build a gigantic model, and select, among the many solutions to your problem, the simplest ones... (I understand this seems impossible to calculate, the "trick" is that so long the model is linear in its parameter, and you can always achieve this in high dimensionality, there are computation workarounds which avoid looking at all those dimensions).

The only thing you should never do is to reduce the dimensions... A bad model might converge, but bad models will be bad.

Francois (sorry for the hijack, people, pet peeve here).

[ilovestrategy]

Herwin, if you do move here to the States permanently where will you go? I'd be honored to meet you.

[herwin]

ORIGINAL: ilovestrategy

Herwin, if you do move here to the States permanently where will you go? I'd be honored to meet you.

We just bought a house in Mariposa, California.

[herwin]

ORIGINAL: fcharton

Hi John,

You've got to get your statboys to read Vapnik... [;)] If they get nosy, just assign them the one published by Wiley (Statistical Learning Theory), real tough stuff...

There are two ways around the problem of dimensionality.

The traditional (Soviet) one is to add artificial constraints to your problem. That's pretty much what regularisation is all about. Say, you have a huge number of parameters, but you add a lot of implicit constraints (typically small norms in parameter space, or stuff like this), which act as a way to force convergence, by selecting one solution among many feasible ones. Basically, you add small "constraint penalties" to the function you minimize, and this stabilizes the calculation.

The modern approach tries to attack the dual problem (a typical aspect of modern maths). Instead of minimising the distance between model and data under constraints, you minimize the constraints in a very large feature space (billions of parameters sometimes happen in practice), under the requiement that distance to data be zero, or below a very small value. In other word, you build a gigantic model, and select, among the many solutions to your problem, the simplest ones... (I understand this seems impossible to calculate, the "trick" is that so long the model is linear in its parameter, and you can always achieve this in high dimensionality, there are computation workarounds which avoid looking at all those dimensions).

The only thing you should never do is to reduce the dimensions... A bad model might converge, but bad models will be bad.

Francois (sorry for the hijack, people, pet peeve here).

Vapnik seems to be a pure theoretician. My head hurts. I need to talk with the local machine learning guru.

[JWE]

ORIGINAL: fcharton
Hi John,

You've got to get your statboys to read Vapnik... [;)] If they get nosy, just assign them the one published by Wiley (Statistical Learning Theory), real tough stuff...
Francois (sorry for the hijack, people, pet peeve here).

I think they have (also sorry for the hijack, but Francois is worth it). I read your post from a while back and asked Jim to mention Vapnik to Tadeusz. I guess there was about 20 minutes of hi-speed technical Polish, followed by Tak, wy odurzony, tak. Now, I've heard tak before, and I've heard odurzony before. Seems brother Tad gets the concept. Think he's thinking ok.

I haven't read Vapnik, and I'm not all that technically proficient in this area. But I know enough to hear what my "Pros from Dover" have to say and let them run with it. My technical expertise is in another area, but I'm bright enough to listen to the man with the big ones.

Ciao. John

[Reconvet]

ORIGINAL: JWE
[ed] But this is such a totally bizarre place to be discussing things like this. It makes me wonder. If there's that much horsepower, why ain't they at the 'Tute AI site? Or Cal Tech's? Or Yale's, or Harvard's, or Chicago's, UCI's. Michigan's, or (damn me for saying it) USC's? There's a lot out there.

Of course specialists can get a higher quality discussion elsewhere, but it's fascinating to get a hint how far AI research has advanced in the last decades, where it's struggling and what might be achieved in near future. Even as a bloody amateur in this area I'm eagerly trying to follow what's coming up in this thread and was motivated to dig deeper into this topic. Thank you guys for the unexpected introduction. [:)]

[herwin]

OK, I've started reading through it, but it still makes my head hurt. There was a reason I bailed out on the math PhD after four years and completed the neuroscience PhD, despite the math being closely related--neuroscience is grounded in real-world examples; math is not. Vapnik's arguments against neural networks and Bayesian inference are formal. He does grant the role of engineering, but he still comments that "neural networks demonstrate surprisingly good results".

I sit on PhD annual reviews, and I've seen a lot of attempts to improve on the performance of the algorithms Vapnik discusses. Many students have tried compromise approaches where they use more than one algorithm and switch algorithms based on the characteristics of the problem. They rarely come up with anything that works any better. Instead, they discover that problems that are hard for one algorithm are hard for all. The whole field seems wrong-headed; none of the current approaches do as well as biological intelligence. And we don't know why.

[fcharton]

ORIGINAL: herwin
How does it happen so fast in real brains? Yes, the experience can be replayed at 20x real-time during sleep, but we see single-presentation learning of odors in rats and rabbits without sleep. We also see the formation of complex concept assemblies in human brains in a very few days. Learning in vivo is very fast.

Two reasons I can think of.

1- animal or human brain make use of specialised sensors (aka our five senses), that preprocess the information into some form amenable to learning. That's a problem with AI: there basically are two approaches :
- learning blobs, ie neural nets or similar, that learn every thing from scratch
- specialized programs, that are tailored for a specific task but never learn
In the living, those two approaches coexist, the brain using "expert sensors" to collect data.
2- you probably never train from scratch, and our brains are probably created with "factory parameters" that are not random. In the heydays of Neural Nets, some people had noticed that, when you trained a network on a specific problem, and then switched to another, the first one tended to be forgotten, but if you switched back training to the first problem, then learning was much faster than the first time.

Francois

[herwin]

ORIGINAL: JWE

ORIGINAL: fcharton
The difficulty, then, is to define a metric to classify events (and refine it using available data). At this point, there is a bad trade off, sometimes known as Bellman's curse.

- to correctly classify complex datasets, you need to take a lot of parameters into account,
- this means your distance (and your datasets) will "live" in a high dimension phase space
- but... the higher the dimension, the further apart two average data points,
- which makes it difficult to find similarities... and therefore learn...

Finding ways to build efficient algorithms in high dimensions is pretty much the subject of modern research on machine learning.

Hi Francois; That's pretty much where LTI is, as you may have guessed from the materials I sent you. We started very simple, with only an 18 element parametric space, but as soon as we expanded them upwards, convergence was problematic, to say the least. There seems to be an inherent instability in an open model. It's almost as though learning must be constrained to a rather truncated set of conditions, with proportional limitations. And that's yet another layer. Woof!!!

[ed] But this is such a totally bizarre place to be discussing things like this. It makes me wonder. If there's that much horsepower, why ain't they at the 'Tute AI site? Or Cal Tech's? Or Yale's, or Harvard's, or Chicago's, UCI's. Michigan's, or (damn me for saying it) USC's? There's a lot out there.

Technical advance is driven by real-world problems. The tendency in math is for the context to be forgotten. That's why the real explosion of developments in cohomology and category theory during the last forty years has been in physics. I go to a lot of meetings on mathematical intelligence, and most of the papers begin by assuming a leaky integrate and fire neuron and then going off to show how it solves some simple problem. My papers begin with a discussion of the real-world neurons I work with--most of which are _not_ leaky integrate and fire neurons--and then explore how networks of those neurons solve hard problems. So this is not a bizarre place to find a real-world problem that needs solving. We're in von Neumann's tradition--he did invent Game Theory with Morgenstern.

[herwin]

ORIGINAL: fcharton

ORIGINAL: herwin
How does it happen so fast in real brains? Yes, the experience can be replayed at 20x real-time during sleep, but we see single-presentation learning of odors in rats and rabbits without sleep. We also see the formation of complex concept assemblies in human brains in a very few days. Learning in vivo is very fast.

Two reasons I can think of.

1- animal or human brain make use of specialised sensors (aka our five senses), that preprocess the information into some form amenable to learning. That's a problem with AI: there basically are two approaches :
- learning blobs, ie neural nets or similar, that learn every thing from scratch
- specialized programs, that are tailored for a specific task but never learn
In the living, those two approaches coexist, the brain using "expert sensors" to collect data.
2- you probably never train from scratch, and our brains are probably created with "factory parameters" that are not random. In the heydays of Neural Nets, some people had noticed that, when you trained a network on a specific problem, and then switched to another, the first one tended to be forgotten, but if you switched back training to the first problem, then learning was much faster than the first time.

Francois

Yes, the brain system does seem to be optimised for learning causal relationships--I mean that's what Hebbian learning does. And you're right about the trainability of neural networks primed for the training. But that means further study of learning (AKA plasticity) in real brains is likely to lead to architectures that learn efficiently. I believe the important features for rapid learning are higher in level than what most researchers model in artificial neural networks.

We have experimental evidence that concept learning involves the formation of cell assemblies made of hundreds of thousands of neurons distributed throughout the brain. A mature concept assembly has a gain of about 1.0--once activated, it stays activated for an extended period. It's hard to build such assemblies in small neural networks.

[fcharton]

ORIGINAL: JWE
[ed] But this is such a totally bizarre place to be discussing things like this. It makes me wonder. If there's that much horsepower, why ain't they at the 'Tute AI site? Or Cal Tech's? Or Yale's, or Harvard's, or Chicago's, UCI's. Michigan's, or (damn me for saying it) USC's? There's a lot out there.

Another reason, not very PC, might be that amateur forums tend to provide more cheerful and relaxed discussion than "pro boards", where everything tends to be serious, class-conscious, and sometimes defensive.

This is not specific to AI or maths, by the way, try a "pro board" about music or litterature anytime...

Francois