Why Everything is more Complicated than it Seems
Posted: Sat Dec 31, 2011 11:04 am
When posting in the war room I had the idea to post a more concrete version here in the tech forum
because this is the location where the latest discussion took place and here is where bugs are reported.
Usually I don´t like to link my own threads, but in this case if you are interested in a constructive
debate, please read it first. The reason is that this post is more or less a continuation of the other.
tm.asp?m=2997103
To point at something specific when talking about the dangers of patching, here comes my favourite
"how do I render a game feature obsolete" story, which has an unsurprising, but also well hidden,
relation to the current "CAP with high number of planes" debate.
Pilot training is borked. Completely. Before you flame me please read to the end.
The only part of WitP AE where I feel the term "completely borked" does justice (so not
some slight inconsistency the average 24/7 forum ranter immediately dubs as "sh**"
but the term "not perfect" is much more apropriate), is the combination of off/on map pilot training.
I do not want to revive a discussion, Matrix Games will not provide the tools/money for a complete
overhaul, and the chance that a radical change with the limited ressources available will make
the situation to worse instead of improving it, is pretty high, based on my experience with similar
situations.
Its just an example, and I think its a pretty good one.
Training is split into two parts, off map and on map training. These parts depend on each other,
both are responsible for a certain task in the training process.
The off map part prepares the ground for the on map training, the on map training completes the
job and releases the pilots into battle or reserve.
There was put much thought into both training models.
Basically the initial intention was to enable a representation of the differences in pilot quality
over the war years by the off map part (so to reflect the situation where the IJN/IJA initial advantge in pilot
quality deteriorates over the years while the Allies increase in quality), and then to enable the player
to reduce this gap partially by on map training.
I think the goal was to provide historical accuracy, but still leave it in the players´ hands to
avoid or pursue historical fates.
Very early in the patch process a forum outcry demanded to change on map pilot training. Many players supported
the opinion that pilot training (on map) was too slow to have the required impact.
These players were mostly Allied players who were the first to experience what being hit by exp/skill 80 elite
pilots feels like, while at the same time being able to field 40-50exp/skill pilots only.
There were a lot of expert opinions stating "pilot training does not take this long in real life", "you gain
knowledge faster in training than in combat", "what is a feature for when the impact is neglectable",
and so on.
It was very soon decided that the root cause of the problem lies in the ammount of time required to train
pilots on map, as well as the exp/skill threshold where the training speed is capped. Opposing opinions
either pointing out other root causes or suggesting that it could also be player related where shut down
quick, and a patch provided the solution: on map training now produces 70/70/70 skill pilots
(e.g. 70exp, 70air, 70def) within 3-4 months.
Not bad. Finally the Allies were able to produce pilots close to Japanese prewar quality with their own
training program and overcome the initial Japanese superiority in experts.
Great is it? Don´t think so.
Here is a list of other potential root causes and side effects the training speed increase had:
1) Other potential root causes (please note that a root cause often is not singular but a combination
of the potential root causes):
A) player behaviour: many Allied players tried to put on resistance in situations where pure situational
analysis should have told them they will lose, because they were in a new situation.
B) the magnifying glass effect: every effect if it is not a closed system in itself has additional variables
at work besides the observed ones.
In this case: the inferiority was caused by worse training of the Allied pilots and the slow on map training,
true, but it was further enhanced by the initially extreme exp/skill of the Japanese side, the better
plane quality of the Japanese side, the numerical advantage of the Japanese side, and the better initial
logistics situation of the Japanese side.
C) The average skill of the Japanese players was generally higher, because it takes more exp to play Japanese,
but also needs more time investement and so dedication to a game. The Japanese player is aware that a working training
routine needs to be implemented in case he wants to see ´45. The average Allied player is not.
D) Exp and skill is modelled into the game with very distinct effects. Low skill has the tendency to result
in failed dice rolls, whereas a skill above the mid-70´s leads to such a high percentage of successful dicerolls.
I think above a certain threshold high exp/skill does not reflect reality in a correct way anymore, but
is rather the abstraction of what in real life would be a combination of very high skill AND unbelievable luck.
Here the system is not variable enough IMHO
2) The side effects of the on map training speed increase:
A) It mothballs the off map training. Completely: The off map training already had its difficulties before the
patch. Off map should provide a basis for on map training, not an alternative to it.
But certain functions in on map training (such as high training effect when exp is low and a slow down when
exp is high) already worked contrary to what was actually desired by the off map training part - which was to
produce a pilot skill curve over the years comparable to the real war.
The speed increase for on map training made off map training obsolete. It does not matter if you harvest pilots with
an average exp of 5-10. The time difference required when comparing to a pilot harvest of avg 40 can be measured in days.
B) The same forum members complaining about the training issue back when they were whacked around as Allied players,
and profited immediately from the patch implementation, now complain that the Japanese player profites from the
same modification and can field uncounted numbers of 70/70 pilots, assign them to high performance plane types,
and keep up resistance in late war never ever witnessed in reality.
C) As described in the potential root cause part, high exp/skill values overpower mission success. Castor Troy tested
and found out that this threshold is somewhere in the 70´s range. Accidentially, thats just about the threshold you can
train your pilots to.
This can lead to an above average number of success rolls, and these range from def skill (pilots are able to evade
close to every attack and so suffer low losses), to navT (5 torpedo runs, 1 hit on average), grdB
(hey, somebody just whacked 100 fighters on the ground, how unfair),...
and enables high number elite strikes to be so homogenous that with good coordination they don´t suffer losses
anymore.
OOOPS
The bottom line is:
The root causes for the initially extreme losses for Allied PBEM players were manifold. One of those root causes
was low pilot quality. But in adressing the pilot quality issue alone by boosting on map training, without paying
attention to the sideeffects ,produced imbalances late war, obsoleted a whole part of the game, and resulted in a
situation where WWII is a combat of veteran against veteran from the beginning to the very end, and that is most
probably an incremental part of the current "my CAP dos not kill as advertised" situation.
I´d really like that everybody who currently runs amok because a certain number of planes is able
to penetrate a high performance CAP umbrella, and everybody who claims that large strike packages often do not
suffer "historically correct" losses, to consider why this might be the case.
Besides a maximum attacks threshold. Because theres a LOT to consider in a game this complex, and its often
not an issue with the simultation but with the input variables (e.g. pilot skill or plane quality).
And if this has been considered, the next thought should be put into the following question:
If I up the maximum attack threshold, will I
- solve the issue described?
- solve an issue that only exists because there are other underlying root causes not adressed by the fix?
- hide potential underlying root causes, up to a point where those aren´t even solvable anymore?
- produce unrealistic results in another situation which is governed by the same variables?
Because if the answer is "all of the above", it is definitely the wrong fix.
I do not want to discourage continuous support and improvement of the game. But please, lets take it
slow. These changes happen too fast and affect too many parts of a system.
If this proceeds it is the sure way to destroy more than is gained. I talk from experience, this is my job.
Thanks for your patience reading this,
and I wish everybody on our forum a happy new year!!
because this is the location where the latest discussion took place and here is where bugs are reported.
Usually I don´t like to link my own threads, but in this case if you are interested in a constructive
debate, please read it first. The reason is that this post is more or less a continuation of the other.
tm.asp?m=2997103
To point at something specific when talking about the dangers of patching, here comes my favourite
"how do I render a game feature obsolete" story, which has an unsurprising, but also well hidden,
relation to the current "CAP with high number of planes" debate.
Pilot training is borked. Completely. Before you flame me please read to the end.
The only part of WitP AE where I feel the term "completely borked" does justice (so not
some slight inconsistency the average 24/7 forum ranter immediately dubs as "sh**"
but the term "not perfect" is much more apropriate), is the combination of off/on map pilot training.
I do not want to revive a discussion, Matrix Games will not provide the tools/money for a complete
overhaul, and the chance that a radical change with the limited ressources available will make
the situation to worse instead of improving it, is pretty high, based on my experience with similar
situations.
Its just an example, and I think its a pretty good one.
Training is split into two parts, off map and on map training. These parts depend on each other,
both are responsible for a certain task in the training process.
The off map part prepares the ground for the on map training, the on map training completes the
job and releases the pilots into battle or reserve.
There was put much thought into both training models.
Basically the initial intention was to enable a representation of the differences in pilot quality
over the war years by the off map part (so to reflect the situation where the IJN/IJA initial advantge in pilot
quality deteriorates over the years while the Allies increase in quality), and then to enable the player
to reduce this gap partially by on map training.
I think the goal was to provide historical accuracy, but still leave it in the players´ hands to
avoid or pursue historical fates.
Very early in the patch process a forum outcry demanded to change on map pilot training. Many players supported
the opinion that pilot training (on map) was too slow to have the required impact.
These players were mostly Allied players who were the first to experience what being hit by exp/skill 80 elite
pilots feels like, while at the same time being able to field 40-50exp/skill pilots only.
There were a lot of expert opinions stating "pilot training does not take this long in real life", "you gain
knowledge faster in training than in combat", "what is a feature for when the impact is neglectable",
and so on.
It was very soon decided that the root cause of the problem lies in the ammount of time required to train
pilots on map, as well as the exp/skill threshold where the training speed is capped. Opposing opinions
either pointing out other root causes or suggesting that it could also be player related where shut down
quick, and a patch provided the solution: on map training now produces 70/70/70 skill pilots
(e.g. 70exp, 70air, 70def) within 3-4 months.
Not bad. Finally the Allies were able to produce pilots close to Japanese prewar quality with their own
training program and overcome the initial Japanese superiority in experts.
Great is it? Don´t think so.
Here is a list of other potential root causes and side effects the training speed increase had:
1) Other potential root causes (please note that a root cause often is not singular but a combination
of the potential root causes):
A) player behaviour: many Allied players tried to put on resistance in situations where pure situational
analysis should have told them they will lose, because they were in a new situation.
B) the magnifying glass effect: every effect if it is not a closed system in itself has additional variables
at work besides the observed ones.
In this case: the inferiority was caused by worse training of the Allied pilots and the slow on map training,
true, but it was further enhanced by the initially extreme exp/skill of the Japanese side, the better
plane quality of the Japanese side, the numerical advantage of the Japanese side, and the better initial
logistics situation of the Japanese side.
C) The average skill of the Japanese players was generally higher, because it takes more exp to play Japanese,
but also needs more time investement and so dedication to a game. The Japanese player is aware that a working training
routine needs to be implemented in case he wants to see ´45. The average Allied player is not.
D) Exp and skill is modelled into the game with very distinct effects. Low skill has the tendency to result
in failed dice rolls, whereas a skill above the mid-70´s leads to such a high percentage of successful dicerolls.
I think above a certain threshold high exp/skill does not reflect reality in a correct way anymore, but
is rather the abstraction of what in real life would be a combination of very high skill AND unbelievable luck.
Here the system is not variable enough IMHO
2) The side effects of the on map training speed increase:
A) It mothballs the off map training. Completely: The off map training already had its difficulties before the
patch. Off map should provide a basis for on map training, not an alternative to it.
But certain functions in on map training (such as high training effect when exp is low and a slow down when
exp is high) already worked contrary to what was actually desired by the off map training part - which was to
produce a pilot skill curve over the years comparable to the real war.
The speed increase for on map training made off map training obsolete. It does not matter if you harvest pilots with
an average exp of 5-10. The time difference required when comparing to a pilot harvest of avg 40 can be measured in days.
B) The same forum members complaining about the training issue back when they were whacked around as Allied players,
and profited immediately from the patch implementation, now complain that the Japanese player profites from the
same modification and can field uncounted numbers of 70/70 pilots, assign them to high performance plane types,
and keep up resistance in late war never ever witnessed in reality.
C) As described in the potential root cause part, high exp/skill values overpower mission success. Castor Troy tested
and found out that this threshold is somewhere in the 70´s range. Accidentially, thats just about the threshold you can
train your pilots to.
This can lead to an above average number of success rolls, and these range from def skill (pilots are able to evade
close to every attack and so suffer low losses), to navT (5 torpedo runs, 1 hit on average), grdB
(hey, somebody just whacked 100 fighters on the ground, how unfair),...
and enables high number elite strikes to be so homogenous that with good coordination they don´t suffer losses
anymore.
OOOPS
The bottom line is:
The root causes for the initially extreme losses for Allied PBEM players were manifold. One of those root causes
was low pilot quality. But in adressing the pilot quality issue alone by boosting on map training, without paying
attention to the sideeffects ,produced imbalances late war, obsoleted a whole part of the game, and resulted in a
situation where WWII is a combat of veteran against veteran from the beginning to the very end, and that is most
probably an incremental part of the current "my CAP dos not kill as advertised" situation.
I´d really like that everybody who currently runs amok because a certain number of planes is able
to penetrate a high performance CAP umbrella, and everybody who claims that large strike packages often do not
suffer "historically correct" losses, to consider why this might be the case.
Besides a maximum attacks threshold. Because theres a LOT to consider in a game this complex, and its often
not an issue with the simultation but with the input variables (e.g. pilot skill or plane quality).
And if this has been considered, the next thought should be put into the following question:
If I up the maximum attack threshold, will I
- solve the issue described?
- solve an issue that only exists because there are other underlying root causes not adressed by the fix?
- hide potential underlying root causes, up to a point where those aren´t even solvable anymore?
- produce unrealistic results in another situation which is governed by the same variables?
Because if the answer is "all of the above", it is definitely the wrong fix.
I do not want to discourage continuous support and improvement of the game. But please, lets take it
slow. These changes happen too fast and affect too many parts of a system.
If this proceeds it is the sure way to destroy more than is gained. I talk from experience, this is my job.
Thanks for your patience reading this,
and I wish everybody on our forum a happy new year!!
