Experience vs Machine Performance

[TheElf]

I'm a fighter pilot. Experience matters more than, top airspeed, maneuver, durability, or training

v/r
Elf

[TheElf]

I've trained new pilots from the day they check into primary. I've also trained them when they first get to a tactical squadron. One big thing they have in common is a pretty significant lack of situational awareness. When they are low time guys they can think, speak, or fly. Don't expect them to do more than one thing at a time! As they become more comfortable in model they can devote more attention to what is going on around them. I'm sure combat will expand your SA (or kill you) but I think its development is a function of building time in your aircraft. A important by product of follow on training at the RAG is expanded SA, but it is a skill that continues to grow throughout your career.

Daniel,
Couple of questions. When did you retire? What did you fly in the Fleet? Where were you stationed in the TRACOM, and when?

v/r
Elf

[TheElf]

I'm a fighter pilot. Experience matters more than, top airspeed, maneuver, durability, or training

I'll let you all digest this for a little while, and then quantify, qualify, and all that later

Elf

[Mike Scholl]

I'm a fighter pilot. Experience matters more than, top airspeed, maneuver, durability, or training v/r
Elf

Thank you for adding an athoritative "voice of experiance" to the thread.
So what it basically boils down to is that if 2by3 "fixes" their air-to-air model
to bring "experiance" up to at least the importance they give aircraft perfor-
mance, we can have a workable game. Mogami seems to have proved in his
testing that Japans "experiance" levels will deteriorate steadily and virtually
unstoppably as the war continues, while Allied squadrons will get remorslessly
better. So if the model is "fixed", then starting the Japanese with an "edge"
in experiance against the majority of their opponants based on the "China
Affair" will produce a good chance for them to duplicate their opening successes
and at the same time guarantee that the "edge" is a wasting asset. Sounds
like a plan to me. MATRIX?

[mdiehl]

Mike, it's not obvious that the model needs fixing. Mogami gets ballpark-realistic results in his WitP alphas. If one enhances the experience factor in the combat model, then one will have to increase USN and USAAF (or decrease Japanese) experience ratings across the board and re-test.

[Mike Scholl]

Mike, it's not obvious that the model needs fixing. Mogami gets ballpark-realistic results in his WitP alphas. If one enhances the experience factor in the combat model, then one will have to increase USN and USAAF (or decrease Japanese) experience ratings across the board and re-test.

I was refering to his testing of equal numbers of various aircraft with different
experiance levels. The point that became rapidly obvious in those was that
the "aircraft stats" got far greater "weight" than the pilots skill. The general
agreement was that that "wasn't right", and needed "fixing".

His early game starts do seem to be producing relatively believable results---
but I suggest that the factor above may indicate that they are occurring for
the wrong reasons, and that play deeper into the War will become skewed
because of it. Nobody's going to know for sure until testing goes farther into
the game. But hopefully it will be tested thoroughly.

[mdiehl]

Ah. Well, when extremes are considered I guess that's true. The "20 EXP" corsairs winning every fight was an odd result. In retrospect I am not sure how odd, however. Since no one seems able to tell me the basis by which these EXP scores were originally assigned it is difficult to know what, exactly, the difference between EXP 20 and EXP 90 is suppose to represent.

The danger here is that giving weight to small differences in EXP (fewer than 30 point differences) will result in combats skewed in favor of higher EXP despite aircraft. A 60 EXP pilot, if that represents a well-trained naval grad with 500 hours of air time, flying an F4U, should usually trounce any pilot in an A6M2.

[mogami]

Hi, I know it has been suggested that US/Allied pilots in UV are under rated. Anyone who cares to load up scenario 19 will find on turn 1 there are on map over 100 Allied pilots rated over 70. (60 is the max you can train a pilot to)
Most Japanese Daitai arrive below full strength. So even if they arrive with pilots over 70 the rest of group will be below 60. It is incorrect to state that all Japanese pilots are rated higher then Allied pilots. There are Japanese pilots in 90's. There are Allied pilots in 90's. These Allied pilots have never been in combat.


Allied pilots in May 1942 (101 fighter pilots over 70 )

35th FS 83,79,76,75,75,72,72,72
36th FS 84,83,75,73,72,72
39th FS 94,88,83,83,83,83
40th FS 85,77,77,77,73,73,72,72,71,71,70
No 75 Sqdn 90,87,85,83,83
No 76 Sqdn 84,74,73,72,71
VF-3 90,86,85,85,85,85,84,83,83,83,83,83,72
VF-42 84,84,84,82,81,76,75,74,74,74,73,73,73,73,72,72,72,71,71,71
VF-6 94,89,88,85,85,84,84,83,83,83,74,73,73,73,73,73,73,73,72,72,72,72,72,72,72,71,71

[mdiehl]

The EXP values don't mean much until one sees how they affect combat. If the importance given to EXP is changed, and if one takes the 11th AF, bases it in Rabaul, and pits it against garden variety USN and USMC units in Guadalcanal I expect to see the fighters fight to about a draw. If the Japanese win disproportionately then the changes to the importance of EXP have resulted in a broken model, and it will have to be corrected by narrowing the gap between the Japanese and their opponents, regardless of the actual numbers assigned to pilots in the pipeline.

The decline happens in the field. Not from dilution as a result of inferior replacements. High EXP Japanese fighter pilots should routinely be shot up and shot down when facing average USMC fighter pilots at a rate of about 1.1:1 favoring the Japanese, and at a rate of about 1.5:1 favoring the US when only CV combats are considered. To the extent that the model does not produce these results, it will be flawed.

[Mike Scholl]

The danger here is that giving weight to small differences in EXP (fewer than 30 point differences) will result in combats skewed in favor of higher EXP despite aircraft. A 60 EXP pilot, if that represents a well-trained naval grad with 500 hours of air time, flying an F4U, should usually trounce any pilot in an A6M2.

While I wouldn't consider 30 points a "small difference" when 55-60 represents
well-trained but not combat experianced replacements, An F4U in reasonable
"equal" hands should certainly have a fair edge over an A6M2. If the Japs
happened to have a very experianced squadron (say average 80) and the
F4U squadron was newly deployed (say ave. 60) then I would expect some-
thing like equality. The better performing A/C will keep the US from getting
"trounced", and the edge in combat experiance would keep the Japs from
getting "trounced". For about a month. Then the better quality of the US
replacements is going to allow the Americans to improve into the 70's while
the lack of well-trained replacements is going to drag the Japanese down
into the 60's. At that point it starts getting more one-sided. Point being
that even an "even" exchange will allow the US to wear the Japs down to the
point where un-even exchanges become the norm,

[mdiehl]

Are you suggesting that the F4F stats will be revised to be better than the A6M2s such that the losses will be approximately equal for one month, with Japanese losses increasing (proportionately) thereafter.

The whole EXP was GGPW's most glaring flaw. Tried many experimnents with that game to duplicate the Guadalcanal historical kill ratios and was wholly unable to do so. Indeed, Japanese group EXP actually increased while US group EXP decreased.

I don't care if the whole thing is called "ham and eggs factor." The bottom line is if I see markedly skewed (ahistorical) loss ratios then the model is just plain wrong.

It's a more critical issue for CV combat. If it takes one month of combats for the USN to get the edge then the USN is going to lose lots of CVs vs few Japanese ones. In point of fact USN pilots should start with a marked advantage over IJN ones.

[mogami]

Hi, I just wanted to make sure people understood it is not a question of Japanese 90's facing Allied 50's. The Allies have many skilled pilots. After
Dec 7th 1941 there is only one path for a pilot to gain exp over 60. He has to fly missions. (A pilot does not have to score kills to gain, he does have to fly non training missions. )

Having low rated pilots (below 60) in a group makes it much easier to kill the higher rated pilots in the same group. When the group faces equal number of fighters and the enemy are all rated higher then a good portion of the pilots in the group the higher rated pilots have to do most of the work. (and the odds are against them)
If all pilots in the game began at 50 then the side that won the majority of early battles would have an advantage in later combats. But since the Allies have pilots (a good number of them) that begin with higher rating then many Japanese pilots they will gain exp. The higher rated Allied pilots are not at as large a disadvantage when compared to the majority of Japanese pilots. (Allied 80 is minus 10 versus the few Japanese 90's but plus 30 or more compared to bulk of Japanese pilots. )

There is no Japanese bias unless it is felt that if top USN/Allied pilots with only peacetime flying and rated in 80's are just as good as Japanese pilots with twice as much peactime flying and in many cases an equal number of combat hours as Allied pilots have total. (Japanese pilots have 700 hours training and 300 hours combat while USN pilots only have the 300 hours training, yet there is only a 10 point exp differance.) I don't think any one can say the Japanese are there for 'overrated'

VF8's peace time ratings
88,87,87,85,85,84,84,84,84,83,83,83,83,83,77,76,76,75,75,75,74,74,74,74,74,
74,74, 9 more pilots 73 and 72
The entire group begins with ratings over 70. This group begins the War (UV) as one of the top groups. (Its better then Shokaku, Zuikaku fighter groups. Better then all the landbased Daitai with the exception of two. And on top of this the Japanese groups (all but 1 ) come on to map under strength (so the new pilots will be below 60)

A group with 13 pilots from 95-82 and 14 pilots from 60-20 is no match for VF-8
(unless the allied player manages to lose at least 12 of the pilots before the battle.) It has been my experience in UV for the USN pilots to always match or exceed the Japanese pilots in air to air combat. They may not shoot down all the higher Japanese pilots in early exchanges but they do gain more exp. (The USN 70's kill IJN 50's and become USN 80's. The IJN 50's are replaced by 40's 30' 20's. This results in more Japanese 90's being killed. After only a few battles the USN groups have climbed 5-10 points in overall experiance while the Japanese groups have dropped 10-20 points. The only counter the Japanese have to this process is introducing more Daitai to the combat. If the allied player see's fit to always allow his groups to be out numbered then he will see them decline. (The Japanese groups will also decline but if he can spread the loss out among more groups it is less noticable.)

The USN has their organizational advantage. USN CV groups contain 36 ac. Japanese CV groups contain from 21 to 27 AC. This in conjuction with the very good USN pilot ratings make USN CV very powerfull. In 1-1 encounters they almost always win. The Japanese need to bring at least twice as many carriers to a battle to break even.

(In my test. Lexington alone took on and defeated both Shokaku and Zuikaku and this with undersize groups.)

[Mike Scholl]

Are you suggesting that the F4F stats will be revised to be better than the A6M2s such that the losses will be approximately equal for one month, with Japanese losses increasing (proportionately) thereafter.

The whole EXP was GGPW's most glaring flaw. Tried many experimnents with that game to duplicate the Guadalcanal historical kill ratios and was wholly unable to do so. Indeed, Japanese group EXP actually increased while US group EXP decreased.

I don't care if the whole thing is called "ham and eggs factor." The bottom line is if I see markedly skewed (ahistorical) loss ratios then the model is just plain wrong.

It's a more critical issue for CV combat. If it takes one month of combats for the USN to get the edge then the USN is going to lose lots of CVs vs few Japanese ones. In point of fact USN pilots should start with a marked advantage over IJN ones.

WHAT F4F? We were discussing and F4U and an A6M2. Where did you find a
reference to an F4F?

If you want to talk about F4F's and A6M2's, fine. Let's talk about them. For
my money they should be relatively equal except for the Zero's extrordinary
range. Both have advantages and disadvantages vis. a vis. one another.
Across a "spread" of missions, all other things being equal, I'd expect a pretty
even "kill ratio" US Navy pilots were well-trained and good (the ones that
weren't wouldn't survive many "controlled crash" carrier landings). Japanese
Navy pilots were well-trained and good (same reasons). Some of them had
had actual combat experiance in China, which would give them a bit of an
edge at the start of the war over the US. But by May-June, the Japs are
going to be running short on trained replacements (according to Mogami), and
that edge is going to be brittle. And historically, while there hadn't been any
carrier clashes, the US had been picking up some experiance raiding various
Japanese islands. Coral Sea was pretty even, and Midway hard to charicterize.
Japanese CAP "did a number" on everything it engaged there, but the US Dive
bombers came over the top and did a job on the carriers. Can't fault the
Zeroes for that, or the pilot rating. The US CAP was able to engage both
strikes on the Yorktown, but unable to prevent them from hitting her. And those
were pretty small strikes.., but made by very determined and experianced air-
crews. The ratio of hits per attack was very high.

The real gist of your question seems to be the Solomons operations. And I think
that with the Japanese having an historical loss rate, they are generally going
to come off second best a bit by then. While they still have some good
squadrons, they're going to have to start bringing in those lower rated semi-
trained fellows to keep the numbers up. The US will still be bringing in 60's.
And that difference of 5 or 10 or 15 each in every new aircraft/pilot brought
in is going to give the US an advantage first in their "equal" aircraft, then in
their lesser aircraft, and finally, when the new designs arrive, a big edge with
their superior aircraft. That sounds about right to me..., how about you?

[HMSWarspite]

People keep asking what the experience levels mean, and I haven't seen it spelled out in one place (forgive me if it has). However, we can deduce what it ought to mean, and hence raise some questions.

100 must mean either the greatest pilot ever (in RL - interesting debate who that would be!), or the theoretical maximum (which no one has necessarily ever reached, or would reach - on the ground that we're all human, and 100 exp pilot doesn't make mistakes).

90 appears to be your 'good ace' - the highest you realistically meet
80 is something like a common, or garden ace
70 is fully trained with a bit of combat experience - maybe a kill or 2 even.
60 fully trained

Now below here it gets interesting. Obviously they are less than fully trained, but what represents the lowest skilled pilot of the war 0? Or 20? It depends on how the scale works. If it is a multiplier on something, 0 would not work, because the net skill or whatever is 0, and thus lowest might be 20. If the scale is not a multiplier in an algorithm, anything could do.

Either way, 30,40,50 are degrees of more training, either off line, or on the job. The reason why I ask, is this. The 'best' aces have 30 points on Joe Average trained pilot. Joe Average trained pilot has ? (40?, 60?) points on Fred 'can't' fly and talk at same time' Kawasaki. Is this real? (Both in training time, and effect on combat). I don't know. In addition, I don't know if 20 points better above rookie is worth the same as 20 points above average trained. Does anyone know these things?

If my scales above are OK, the first kill should be worth maybe 5 exp, falling away such that kills 5-10 get about 5-10 exp between them. Conversely, the first fight for a 60 pilot is worth say 2 exp (no kill), but possibly 10 exp for a green 20 exp guy. Still hopeless, but now been there once!

From what I have read, I would agree that situational awareness is the real divider between success and a statistic. I believe that SA can be taught (more so today with replays, lasers, radar plots etc). How long did it take to gain in WW2 training...? I don't know - maybe it could not be effectively taught, in which case 60 exp pilots should still be at risk, but 70 exp are ok. What I do know is that the average product of US or German (and UK for that matter) has something like 1/10th the loss rate in engagements 6 on to that in the first decisive engagement. Does the experience system produce this effect with 60 point pilots (or do you get it with 50, or 40)? For the record the peak loss rate tends towards 0.4% for engagement 1 Ge pilots, and 0.3% mission 1 US (I suspect late war Ge lack of training might explain the difference). You have also to take in to account the quality of the opposition, but this might imply that against average 60 exp pilots you should lose 0.3-0.4% of your newbies in their first decisive engagement. After this, the experience gain should drop losses to an almost flat 0.01% per engagement.

Has anyone tracked a squadron through multiple controlled engagements? (Tricky, I do not know how you would, because the enemy keeps changing - I guess you could define 10 identical enemy groups, and fly the target group against each in turn?). The statistics of interest would be number of missions flown by each casualty. You would probably need to run this a lot!

My thoughts on how experience should work give the effect as something like this:

Green, exp <40 (or something) - gets shot down by almost anything flown competently. Stuggles to shoot anything down...ever! (regardless of a/c).
Some training (40-50) loss rate of say 0.3-0.4% per engagement (for pilots in this skill group, not total), regardless of a/c, occasionally shoots something down if the enemy is a/c is inferior, or badly flown
Trained 60. A/c will tell when fighting equal exp, otherwise still in the 0.3-0.4% bracket, can shoot things down if in better a/c, or badly flown
Above 70, starts to get kills regardless of being in a worse a/c, especially against inferior exp. The higher the exp, the more often will get kills, despite a inferior a/c.

Of course, both of these interact, so you can get:

Thus Corsair (20) vs A6M3 (80), likely to be a fair few Corsairs lost, very few if any Zeros (Corsairs have no SA, and struggle with the flying!)
Corsair (20) vs A6M3 (20) - nothing happens big time!
Corsair (50) Zero (50), Zeros some losses, maybe a few Corsairs (not many)
Corsair (50) Zero (70) bit of a stand off - Zeros know not to do anything silly, Corsairs don't make too many mistakes
Corsair (70) Zero (70) - Zero starts to lose some (Corsairs know how to exploit the a/c)
Corsair (80) Zero (30) - Zero loses big time (huge in fact)


The worse the aircraft in capabilty, the better the pilots need to be to achieve results, so
A5M (90) vs Corsair (30) - might get some Corsairs
A5M (70) vs Corsair (30) - struggle
A5M (70) vs Corsair (40) - stand off (few losses)
A5M (70) vs Corsair (50) - start getting IJN killed (instead of my 'bit of a stand off above)


This is all very subjective, but my 2p worth...

[mogami]

Hi, Unless the starting Japanese IJN pilot pool gets bumped up (from 100) the Japanese will be out of trained replacements before the end of Dec 41.
Then they have to keep loss below 10 a month for rest of war to avoid using untrained pilots. (If they get a new group with more then 10 pilots it will be untrained)

[mdiehl]

.... and Midway hard to charicterize. Japanese CAP "did a number" on everything it engaged there, but the US Dive bombers came over the top and did a job on the carriers.

What are you talking about? The only aircraft that the Japanese CAP "did a number on" at Midway were unescorted, obsolete, single-engined strike a/c.

The reason why I shifted the discussion from the particular (F4U vs A6M2) to the other particular (F4F vs A6M2) is because if one changes either the EXP ratings or else the underlying arithmatic that implements these in the code to increase the importance of EXP, THEN the early war combats between F4Fs and A6M2s are going to be skewed from where they presently stand. According to Mogami's WitP alpha test AARs, the early war model for A6M2s vs F4Fs seems to produce pretty realistic results. If one then changes the general model to favor the (generally higher rated EXP-wise) Japanese, then these early war combats are going to favor Japan and thus throw off the present rather accurate results.

I honestly don't give a rip about how long it takes for the Japanese pilot pool to deteriorate. If it results in a series of actions over say, one month, that greatly favor the Japanese in the early war, it's gonna be a broken model. if it results in a situation in which the USN player must be risk averse because he can expect to be beaten in confrontations with comparable numbers of IJN CVs, it's gonna be a broken model. Period. One can argue that "after all the Allies can correct for this if only they do so and so (like "avoid carrier combat" or "develop a tolerance for ahistorically high loss rates knowing that eventually he's gonna wear the opponent down") the model will be wrong. More than that, it will most definitely not result in a game that puts the Allied player in a position where the game that he's playing "looks and feels" like the real thing to the extent that he is concerned with the same issues and willing to take the same risks.

[mdiehl]

Thus Corsair (20) vs A6M3 (80), likely to be a fair few Corsairs lost, very few if any Zeros (Corsairs have no SA, and struggle with the flying!)
Corsair (20) vs A6M3 (20) - nothing happens big time!
Corsair (50) Zero (50), Zeros some losses, maybe a few Corsairs (not many)
Corsair (50) Zero (70) bit of a stand off - Zeros know not to do anything silly, Corsairs don't make too many mistakes
Corsair (70) Zero (70) - Zero starts to lose some (Corsairs know how to exploit the a/c)
Corsair (80) Zero (30) - Zero loses big time (huge in fact)

I do not agree. Here's how I see it:

Corsair (20) vs A6M3 (80): a couple of F4Us shot down, high operational losses for the F4Us, low operational losses for the A6M2s, maybe a Corsair gets lucky and takes down a Zeke.

Corsair (20) vs A6M3 (20): A few Zekes shot down. No F4Us shot down unless a Zeke gets very lucky. High operational losses all around.

Corsair (50) Zero (50): Zeroes take a pasting, losing three for every Corsair brought down. Operational losses "normal" if "50 EXP" is assumed to be "normal training."

Corsair (50) Zero (70): Slightly more Zeroes shot down than Corsairs. Operational losses for the Corsairs normal. Slightly lower op losses for the Zekes.

Corsair (70) Zero (70): Zeroes losing big time. Five or six to one destroyed for each F4U brought down. Operational losses low.

Corsair (80) Zero (30): Zeroes horridly trounced in a bloody 1-sided, no-contest engagement, losing dozens of A6M2s for each F4U brought down. Post combat, most of the remaining damaged Zekes are lost operationally. Japanese ground crews weep as strikes of hundreds of aircraft see fewer than a score returning to their base(s). Air combats see American pilots making ace in a single combat, sometimes seeing interceptions where modest numbers of Corsairs destroy several times their number in enemy fighters, break through all escorts, shoot down most of the bombers, and for the most part return safely to their bases. At night, on Japanese airfields, the ice-weasels come and finish off whatever's left.

[Daniel Oskar]

Daniel,
Couple of questions. When did you retire? What did you fly in the Fleet? Where were you stationed in the TRACOM, and when?

Elf

Hey Elf, I retired 1 Oct '01 (my ceremony was on 10 Sept) My last fleet tour I flew the CH-46E out of MCAS New River NC. My last tour was with VT-2 at NAS Whiting as the Stan Officer from Aug '97-Oct'01. It was a long tour 'cause they were kind enough to let me stay there until retirement.

[denisonh]

At what point and to what extent does the Squadron Commander and component bias (IJA vs IJN vs USAAC vs USN vs USMC vs RAAF vs RNZAF etc...)affect the air combat?

I would think that there must be a more significant effect from the squadron commander and air component tactics and techniques than from an individual pilot.

Is it fair to say that early war USAAC P-40s performed as well as their AVG counterparts? I am not sure it is, and would venture to say that leadership and tactics had a much greater effect on the outcome than simple pilot skill or experience and "generic" aircraft performance.

Training and doctrine applied by the squadron leadership seem to be critical aspects of air combat that should be appropriately modeled. Taking advantage of pilot and aircraft strengths while at the same time minimizing the effects of pilot and aircraft weaknesses through the application of tactics and techniques is a function of leadership and training.

So some effect based on component (over time) and squadron leadership (if not already modeled) may be a better way to approach this problem.

[mogami]

Hi, There was a German pilot training Japanese pilots in German (and Allied) tactics in Japan up to Mar 1942. (Stationed in Tokyo in late 1940 early 1941)
The Japanese were Claude groups but all these groups were upgraded to A6M2 before leaving Japan. Don't know how much help it would be later but it seems a large number of IJN fighter pilots had been trained in the allied tactics current up through the Battle of Britian.


Also it appears the Chinese Airforce was rebuilt at least three times by shipments of Western aircraft. So the IJN pilots had faced almost every fighter they later encountered with the exceptions of P-40 and F4F and Hurricane.

They had fought all the Soviet types as well. (I think the A6M2's fighting qualities were in response to I-16 among others)
The IJN pilots did their flying in Type 96 Claudes and still defeated Chinese pilots in Western and Soviet fighters.