Pilot Training Test

[SeethingErmine]

The game-within-a-game aspect of pilot training has had my attention lately, so I fired up the editor and setup a testing scenario in order to see how some factors play into training results. (The editor is Really nice by the way! [:)])

Two factors that have been identified as affecting training are the number of veterans in a group, and the leadership of the air group leader. Since both of these things have a fairly evident costs or opportunity costs in game (distributing a limited number of veterans and political points), I setup this test to try to get some quantitative assessment of the benefit and ultimate value of these choices.


Test scenario:

Application version 1.0.2.1097 (2010/01/03)
600 planes, 24x25 groups, training Sweep @ 10K feet at 100% over 60 days (3 day turns).
600 trainee pilots generated randomly with average of 35 experience.
54 veteran pilots with 85 experience spread in groups of 1, 3, and 5.
(All groups had 25 trainee pilots, so a group with 3 veterans has 28 pilots total while training.)

Training occurred at level 7 air base with Air HQ and attached Command HQ present at the base.
Base was attached to the Command HQ, and 100,000 supplies was present at base.

All leader skills are 50 (including HQ leaders) unless otherwise stated.

10 planes, 2 pilots lost to OPs.
Veterans remained relatively unchanged after training, although two fell to 84 exp and 4 rose to 86.

Group averages taken after removing veterans (E=experience, A=air skill, D=approximate defense skill):

Air Groups with Leader Leadership Skill at 40 results:

0 Veterans
IJA. 1.1: 43 E, 67 A, 56 D
RAF 1.1: 44 E, 67 A, 55 D

1 Veteran
IJA. 1.2: 44 E, 67 A, 54 D * -1 plane
RAF 1.2: 44 E, 67 A, 54 D

3 Veterans
IJA. 1.3: 44 E, 68 A, 54 D * -2 planes
RAF 1.3: 44 E, 68 A, 56 D

5 Veterans
IJA. 1.4: 44 E, 68 A, 54 D
RAF 1.4: 44 E, 68 A, 54 D


Air Groups with Leader Leadership Skill at 60 results:

0 Veterans
IJA. 2.1: 45 E, 67 A, 58 D * -1 plane, -1 pilot
RAF 2.1: 45 E, 67 A, 54 D

1 Veteran
IJA. 2.2: 45 E, 68 A, 53 D * -1 plane, -1 pilot
RAF 2.2: 45 E, 67 A, 55 D

3 Veterans
IJA. 2.3: 44 E, 68 A, 54 D * -4 planes
RAF 2.3: 44 E, 68 A, 56 D

5 Veterans
IJA. 2.4: 44 E, 68 A, 54 D
RAF 2.4: 44 E, 68 A, 56 D


Air Groups with Leader Leadership Skill at 80 results:

0 Veterans
IJA. 3.1: 45 E, 68 A, 55 D
RAF 3.1: 45 E, 68 A, 55 D

1 Veteran
IJA. 3.2: 44 E, 68 A, 54 D
RAF 3.2: 44 E, 68 A, 57 D

3 Veterans
IJA. 3.3: 44 E, 68 A, 53 D
RAF 3.3: 44 E, 68 A, 56 D * -1 plane

5 Veterans
IJA. 3.4: 44 E, 68 A, 56 D
RAF 3.4: 45 E, 68 A, 54 D



Same data, grouped by veterans present (L=leader leadership)

0 Veterans
IJA. 1.1: 43 E, 67 A, 56 D (40 L)
RAF 1.1: 44 E, 67 A, 55 D (40 L)
IJA. 2.1: 45 E, 67 A, 58 D (60 L) * -1 plane, -1 pilot
RAF 2.1: 45 E, 67 A, 54 D (60 L)
IJA. 3.1: 45 E, 68 A, 55 D (80 L)
RAF 3.1: 45 E, 68 A, 55 D (80 L)

1 Veteran
IJA. 1.2: 44 E, 67 A, 54 D (40 L) * -1 plane
RAF 1.2: 44 E, 67 A, 54 D (40 L)
IJA. 2.2: 45 E, 68 A, 53 D (60 L) * -1 plane, -1 pilot
RAF 2.2: 45 E, 67 A, 55 D (60 L)
IJA. 3.2: 44 E, 68 A, 54 D (80 L)
RAF 3.2: 44 E, 68 A, 57 D (80 L)

3 Veterans
IJA. 1.3: 44 E, 68 A, 54 D (40 L) * -2 planes
RAF 1.3: 44 E, 68 A, 56 D (40 L)
IJA. 2.3: 44 E, 68 A, 54 D (60 L) * -4 planes
RAF 2.3: 44 E, 68 A, 56 D (60 L)
IJA. 3.3: 44 E, 68 A, 53 D (80 L)
RAF 3.3: 44 E, 68 A, 56 D (80 L) * -1 plane

5 Veterans
IJA. 1.4: 44 E, 68 A, 54 D (40 L)
RAF 1.4: 44 E, 68 A, 54 D (40 L)
IJA. 2.4: 44 E, 68 A, 54 D (60 L)
RAF 2.4: 44 E, 68 A, 56 D (60 L)
IJA. 3.4: 44 E, 68 A, 56 D (80 L)
RAF 3.4: 45 E, 68 A, 54 D (80 L)



I should note that in addition to everything that is random about the process of gains, the randomness of the starting pilots may affect the results. I'm sure if I ran this again I'd see minor variations everywhere. I still think this data is suggestive of the impact these decisions have in game.

edit: added version

[CapAndGown]

From what I can see, the idea that veterans help a squadron improve faster or further is a myth.

BTW: Thanks for running the test.

[dorjun driver]

I gotta agree with C&G. These data don't seem to support the "veteran" boost hypothesis.[:(]

[Nomad]

It also doesn't look like the leadership value of the commander has much if any effect.

[topeverest]

This is very helpful. I was trying to run tests in parallel to play not nearly as well designed. This is pretty definitive on the relative difference the value of training has with these paramaters. These results suggest that time spent training and starting values are more important to attribute gains than other factors typically related to training.

To be clear in the event I am being thick, this is the test of non TRACOM training.

[SeethingErmine]

I've run some other tests with truly terrible AG leaders (leadership 10), and the group's experience was noticeably impacted (negatively), so I'm pretty confident there is some correlation. 

My thinking is that past a certain point these factors are just light "nudges" on training improvement, and that either by intent or a result of the mechanism they don't stack significantly.  Notice that the only group that failed to hit 44 average experience had neither a good leader nor a single veteran, while every group with either an excellent leader or 3+ veterans hit 68 air skill.  The results are so close I suppose it could be an illusion, but I've seen a similar distribution (low-end outliers with no help, consistent high values with help) in other runs so I don't think it's just coincidence.

topeverest -
That's right, I was testing different numbers of veterans in the training air group (0,1,3,5) (with varying AG leadership skill).   TRACOM had no veterans.


edit: In all my tests the defense skill seemed very volatile & random. I think because it is not being trained directly it uses different rules. Air skill improvements were much more consistent and repeatable distributions, by comparison.

[Nemo121]

Could you run tests with the leaders having high experience ( but leadership being the same ) ? I suspect that inspiration might play a role here.

Overall though it looks like if you have leaders at the extremes of ability you might just have an impact but otherwise it really is a matter of time. Put the time in and you get the training out.

I must admit a less than 10% variation in outcome for tweaking leaders, veterans etc isn't too impressive to me. Probably not worth the trouble.

[SeethingErmine]

Nemo121 -

I'm confused because leaders don't appear to have experience (in game or in the editor). Am I missing something?


I ran through another full test using the same trainee/veteran compositions, but with varying leader inspiration instead of leadership. Seems clear from the results that neither inspiration (directly) nor morale impact training.

Test scenario:

Application version 1.0.2.1097 (2010/01/03)
600 planes, 24x25 groups, training Sweep @ 10K feet at 100% over 60 days (3 day turns).
600 trainee pilots generated randomly with average of 35 experience.
54 veteran pilots with 85 experience spread in groups of 1, 3, and 5.
(All groups had 25 trainee pilots, so a group with 3 veterans has 28 pilots total while training.)

Training occurred at level 7 air base with Air HQ and attached Command HQ present at the base.
Base was attached to the Command HQ, and 100,000 supplies was present at base.

All leader skills are 50 (including HQ leaders) unless otherwise stated.

Group averages taken after removing veterans,
(E=experience, A=air skill, D=approximate defense skill, M=morale, V=# of veterans during training):


Air Groups with Leader Inspiration Skill at 50 results:

IJA. 1.1: 44 E, 68 A, 56 D (56 M, 0 V)
RAF 1.1: 45 E, 68 A, 55 D (48 M, 0 V)
IJA. 1.2: 44 E, 67 A, 57 D (56 M, 1 V)
RAF 1.2: 45 E, 68 A, 55 D (64 M, 1 V)
IJA. 1.3: 45 E, 68 A, 55 D (66 M, 3 V)
RAF 1.3: 44 E, 67 A, 56 D (99 M, 3 V) * -2 planes
IJA. 1.4: 44 E, 68 A, 55 D (85 M, 5 V)
RAF 1.4: 44 E, 68 A, 54 D (90 M, 5 V) * -1 plane


Air Groups with Leader Inspiration Skill at 10 results:

IJA. 2.1: 45 E, 68 A, 56 D (20 M, 0 V) * -1 plane
RAF 2.1: 45 E, 67 A, 55 D (48 M, 0 V)
IJA. 2.2: 44 E, 68 A, 55 D (43 M, 1 V)
RAF 2.2: 45 E, 67 A, 56 D (44 M, 1 V)
IJA. 2.3: 44 E, 68 A, 54 D (24 M, 3 V) * -1 pilot, -1 plane
RAF 2.3: 44 E, 68 A, 57 D (37 M, 3 V) * -1 plane
IJA. 2.4: 44 E, 67 A, 55 D (59 M, 5 V)
RAF 2.4: 44 E, 68 A, 55 D (59 M, 5 V)


Air Groups with Leader Inspiration Skill at 90 results:

IJA. 3.1: 45 E, 67 A, 56 D (99 M, 0 V)
RAF 3.1: 45 E, 68 A, 56 D (99 M, 0 V) * -1 plane
IJA. 3.2: 44 E, 67 A, 56 D (99 M, 1 V) * -1 plane
RAF 3.2: 45 E, 68 A, 56 D (99 M, 1 V)
IJA. 3.3: 44 E, 68 A, 54 D (99 M, 3 V) * -1 pilot, -1 plane
RAF 3.3: 44 E, 68 A, 55 D (99 M, 3 V)
IJA. 3.4: 43 E, 67 A, 56 D (99 M, 5 V) * -3 planes
RAF 3.4: 44 E, 68 A, 56 D (99 M, 5 V)

[stuman]

Very interesting. Thank you for taking the time for running these tests. It seems that " good enough " leaders and time is all that really matters.

[castor troy]

Hurrey, nothing effects pilots training... lol, sometimes you have to wonder what you are being told before someone actually tests it [:D]

[Chickenboy]

@Seethingermine:
 
First off, I so appreciate your screen name.  Very nice.
 
Secondly, thank you VERY much for taking the effort to do this.  Very helpful and informative.

[SeethingErmine]

@Chickenboy - Thanks! [:)] It's a little silly but I like how it rolls together and breaks apart. "See" or "SE" for short works too.


This isn't exactly training related, but I always wondered if admin skill had any impact on pilot fatigue. (Nothing said it did, but with something this complex you never really know for sure, right?) I ran a test for a bit, and it confirmed zero impact on pilot fatigue all the way up the chain of command (AG, Air HQ, and Comm HQ all at 5 admin performed identically to all 50s).

I also tried to find some correlation between AG leader air skill and A2A results by using 4x25 groups sweeping a set of 4x25 groups on CAP (each at the same altitude). I'd run it for 6 days, then switch to different sets of four (on both ends) using the same conditions but with different leader skill. The loss results were very volatile and I couldn't confirm any association even using 10 air skill leaders vs 90 air skill leaders (randomness and external factors, like weather, dominated, I think).

However, I did reliably see a huge association between sweep cohesion and the leader's air skill. Four different groups of 25 fighters, filled with 90% raw 35 experience pilots swept in as a combined 100 plane raid with all four leaders at 90 air skill. With all four leaders at 10 air skill, I saw five different attacks on the same day a couple times. (Mind you, this is with all four groups flying from the same base to the same target with the same settings (range was 4).) Pretty spectacular loss of cohesion if you ask me. [:)]

[vonSchnitter]

as interesting - or revealing - seethingermine's findings are - hat off btw - some findings got lost.

An Airgroup (AG) on training only will not train predicatbly:

exp is slowest, the specialty is fast annd defensive is slow .

case 1) IF all pilots are in the same band (30s), here goes:
exp 1, specialty 4, defense 2 (or thereabouts)
Case 2) If pilots are of mixed experience/skill

Pilots with higher exp will take longer to gain more exp on training only - as the manual says.
pilots with higher skills in the trained specialty will take longer to train
Pilots with higher exp or skills will take longer in "cross training" - ie. an army air unit with pilot exp in the say 60s and some land bomabardment in the 60s will take longer to "adjust" TO ASW than a group of newbies (30 exp no specialty)
As a result, in mixed AGs (some pilots 60 some 30), low exp pilots aquire the specialty trained at triple the speed of the experienced ones - with gaines in the defensive department as well.

[vonSchnitter]

One more thing:

Mixed AGs (Say half on training, half on active duty (say CAP ) are different again.
Regardles of training settings (or specialty) no pilot will gain defensive points (as in training only ) - but the low exp pilots gain in exp and specialty ...  which is fine.

[vonSchnitter]

From my end, the whole original manual on training etc is just bogus.
The training groups as per the manual got shafted - once introduced after release - for whatever reasons - and the whole issue is in limbo since - at least as far as a "readme" explanation is concerned.

Looks like the "team" has opted for some sort of "complexity by obscurity" solution for now, which is easily eyplained by the attrition rate of original contributors as by this forum

[Schatten]

A Question..i have think Exp 50 is max with Training?
Situation: I have replaced all the Pilots in Kwantung Air Units that can send Pilots into General Pool with fresh Pilots from Replacement Pool and set the Unit...lets say one of the Nate´s to Training Escor 100%

Most of them now over 40 EXP (42-46 from 25-32), some 49 with Air Combat and Deefense over 60 and few have Hit 50 Exp.

Usually i send them at this Point into General Pool but have forgot some...and now one of them hit 51% Exp...the Unit is at russian Border and has see absolutly no action except Training Escort 100%


So how high can Exp going up by Training only (and Air+Defense or GrundBomb)?

[Chickenboy]

@ Schatten: my experiences with training are that skills asymptotically approach 70 (not 50) with time. I'll usuallly start rotating my fliers over to front line units when they're close to that, as additional progress is not timely.

[findmeifyoucan]

I agree with Chickenboy. With training and time you can acquire up to the 70 level of experience. Having at lease a few Veterans helps too!

[Nemo121]

Sorry, I meant Inspiration. I was wondering if inspiration made a difference to training gains.

I've always found that AGGRESSION factored more into kill rates than Leadership as aggression ensured a high number of passes would be made. Leadership seemed to effect the exchange rate shifting it in a slightly favourable direction ( probably more due to better cohesion etc than anything else ).


Overall though I'm not having much trouble filling my front-line fighter groups with 65+ A2A Exp fighter pilots from my on-map training programme in mid to late January 42. I'm anticipating that so long as I don't engage in significant attrition but, instead, choose to fight on mostly favourable terms and only when necessary that I will be able to maintain this +65 A2A cut-off for my front-line fighter groups ( I'm actively rotating out the lowest A2A exp pilots from these groups back into 2nd echelon training groups ). This experience level seems to make a huge difference to outcomes. I have one 7 kill pilot and about 5 or 6 x 4 kill pilots as well as multiple 3 kill pilots with my first purpose-created fighter group ( the 232nd RAF group ) having a 33 to 4 kill rate. This was a unit in which I swapped out all the sub-60 A2A Exp pilots and swapped in nothing but 60+ A2A Exp pilots.

[witpqs]

ORIGINAL: Nemo121

Overall though I'm not having much trouble filling my front-line fighter groups with 65+ A2A Exp fighter pilots from my on-map training programme in mid to late January 42.

Which side?