Pilot Training Automation Tool (PTAT)

[WriterNotViking]

Oh, heck no! I thought you raised a valid question that many inexperienced players might hesitate to ask, and answering it here would serve to underscore the utility of the tool in question.

[richlove]

@Sangeli - very impressive piece of software development. What software did you use?

I agree w/ you - I think it'll be hard (probably impossible) to be able to filter the 'correct' pilot to move in the non-monotonic groups. When I move pilots around, I hold up a ruler to the screen to find the right ones to pull from the Reserve!

Have you considered pulling the pilot skill data from outside the .exe completely (like however Tracker gets its data) and building lists of pilots to move around, then acting on those lists? I have no idea if reading that info is possible... but in my head I can see the database tables I'd create and the queries I'd run to get those pilots.

[Sangeli]

Thanks for all your feedback! I'm actually happy to see the responses from people who think its a bad idea. I'm always interested in counterpoints. Some of them are valid too. Micro-managing pilots just gives you a small edge in a PBEM. Spending lots of time on pilots doesn't guarantee you victory nor does lax pilot management prevent you from achieving it. So I'm not surprised the people who have AARs are also the ones seem more interested in the tool.

ORIGINAL: Roger Neilson 3
The video really doesn't help me at all, its pretty fast and I have no idea what is happening... A bit of audio explanation would help enormously.

Yes, I realize I just sort of threw the video out there without any real explanation. I didn't want to explain too much because I don't expect the final program to work like the one you see beside the mechanics of releasing and adding pilots. All I intended to show in the video was that I have a program that is capable of adding and releasing pilots with some sort of logic behind it. That and throw some ideas out there. Next time I make a video I'll be sure to explain more what is going on. I also have the ability to slow the program down by any arbitrary time scale factor.

ORIGINAL: richlove

@Sangeli - very impressive piece of software development. What software did you use?

I agree w/ you - I think it'll be hard (probably impossible) to be able to filter the 'correct' pilot to move in the non-monotonic groups. When I move pilots around, I hold up a ruler to the screen to find the right ones to pull from the Reserve!

Have you considered pulling the pilot skill data from outside the .exe completely (like however Tracker gets its data) and building lists of pilots to move around, then acting on those lists? I have no idea if reading that info is possible... but in my head I can see the database tables I'd create and the queries I'd run to get those pilots.

Thanks! I am using Python 2.7. Many of my programming solutions are naive and mediocre since I have no prior experience in things like OCR (how the engine recognizing whats going on in the game) but it seems to be doing a good enough job.

Yes, I have considered using the save file to quickly build a database for pilots. In fact that is currently my plan. Eventually I want to have a program that can determine where to put every pilot before a single one is added. To do that I need to know what's available and what is in each squadron. I also have the ability to create this database on the fly by examining the pilot screen in the game itself. Certainly though that will be a lot slower and probably is not suitable to the giant pilot pools; its mostly just for the squadrons themselves.

[witpqs]

ORIGINAL: WriterNotViking

Oh, heck no! I thought you raised a valid question that many inexperienced players might hesitate to ask, and answering it here would serve to underscore the utility of the tool in question.

Right you are, Roger raised a very valid question. If your opponent is studious about pilot training it ups his game.

[Rafid]

When I first saw the video, I started laughing and said "how cool is that?". I would consider such a tool a great help. I think pilot training to be very repetitive and the only aspect of the game which is sub par in terms of return (of strategic depth) of invest (of time and mouse clicks).

I'm guessing you are using autopy for mouse control and PIL (or PILLOW) for screen capture. What are you using for OCR? I can imagine the WITPAE font to be very challenging, due to the low resolution. Is the slight "thinking pause" when the pilots screen is first loaded due to OCR of the skill table?

I think programming an algorithm which mirrors what some of the more advanced micro-managers do, would be extremely difficult.

[witpaemail]

ORIGINAL: Malagant

I think you guys are talking Apples and Oranges.

witpaemail, you're talking about moving trained pilots to fighting groups.

I think what Sangeli is trying to automate is the intense micro-management that many players to do achieve certain sets of skills on pilots, not just a single skill.


Sangeli, to answer your question...I'm not terribly experienced player, but I find I do a whole "class" at a time in a squadron: i work one skill so they're all in the 70s range, then switch to the next skill. I don't micromanage each pilot, I just do a batch at a time and aim for 'pretty good', not perfect

With very few exceptions, a given pilot only really needs 1 skill.

Fighter pilots need air to air, pretty much period. (yes they need defense also, but all training trains defense as well as its "main" task)
Land based bombers for the most part only need land bombing skills.
Naval bombers for the most part only need naval bombing skills.
Torpedo bombers are the exception as they need both torp attack and naval bombing.
Patrols need naval search.
Floats need naval search.
Recon need recon.
Transports need transport.

When my land bombers are up to the 70s, if I dont need them at the front, I will switch their training to something else like naval bombing.
As above, when a squadron has trained into the 70s in its primary, I will switch to a secondary skill for them, IF I DONT NEED THEM at the time.

No real "need" to micro-manage a single pilots training.

Some squadrons I train ASW from the start. All Jap land bombers with a total max number of aircraft 12 or less for example. The Helen 1 makes a good ASW aircraft after they get the MAD, so I train some pilots to put in them.

If A pilots experience is greater than the skill for the task it is trying to perform (example a 80 experience pilot with a 20 naval bombing skill) will perform the task at 1/2 his experience. So in that example, even though his "skill" is only 20, he would attack as a 40 because of his experience.

I also use squadrons to train "other skills". For example, I will use a Navy Kingfisher squadron to train ASW, so I have a handful of ASW pilots in the pool, ect. But now we are getting into the micro managing bit, but as someone else said, once a month or once ever 2 months is generally good enough to handle this.

[Bullwinkle58]

ORIGINAL: Sangeli

Yes, I have considered using the save file to quickly build a database for pilots. In fact that is currently my plan.

You could contact Floyd and Damien directly, but I think they had to go to Matrix/Henderson (?) and do a partial NDA and all those hoops to get access to the save file format. I think it's encrypted? Matrix has always treated the save file as the crown jewels of security. If people can play with it the PBEM portion of their marketing crashes and burns. In a game that takes real-time years to play even a hint that somebody could "edit" the save game file would be a disaster.

[bartrat]

You could use TRACKER data instead of contacting the programmers, you can read the JDBC file I have done it using Crystal reports.

[kbfchicago]

watched the video. Thank you for posting to the community and putting in the time to explore this...

Observations from my foxhole on its use;

- I manage pilots (as someone above said) in "batches", setup training and let'em cook for a 30-90 days, review 1-2 times a month, adjust from current to training on a different skill when "many" (but not all) of the pilots hit an acceptable skill threshold.
- After they hit required skill levels in all areas, either move them en-mass into the Reserve pool or move them from the training squadron into active squadrons via replacement/veteran and sqdn2sqdn xfer. Then restock with noobs and start again.

I find the most ...frustrating is not the right word, nor is annoying, but something in between...tedious maybe, is sorting through the Reserve pool for the right combination of skills. Filling a Ki45 or Beaufighter unit with guys who have reasonable A2A and also have good low ground/nav attack, etc.. Since the Reserve (and unit) Pilot screens only allow you filter on class of pilot then sort on one column I end up doing a sort on the primary skill and manual scan across secondary ones till I can find matches to what I need.

Sorry, long way around to say it's not the training or task of assigning (to me) that's tedious and needs help, its the sorting, especially from the Reserve pool.

Recommendations and feedback;

I would also be hesitant to trust any automated assignment tool (the game already has one I don't use, because I've ended up with transport pilots in fighter groups...) that did not have a "here is the list of what we are going to send to unit x, you good with this?" So if you continue to refine on your idea of finding max qualified & xfering via a script, I'd recommend you group the assignments, then let the player review to confirm.

IF (note caps...) you were to conclude most players leverage the Reserve Pool as a transit point (or would do so with the use of this tool). You may want to just focus on pulling lists of best qualified (by multiple skills) from the Reserve (vs. looking across all squadrons). Then allow players to pick (or at least remove) from the filtered list and execute a transfer of multiple pilots with one click. That would focus your programing (on reserve pool), provide players a means to sort on multiple skills (something not currently in the game), and potentially save a few clicks.

Thanks again, am always impressed by the (many many hours of) work by a few to develop tools that potentially help us all!

Kevin

[Sangeli]

ORIGINAL: Rafid
I'm guessing you are using autopy for mouse control and PIL (or PILLOW) for screen capture. What are you using for OCR? I can imagine the WITPAE font to be very challenging, due to the low resolution. Is the slight "thinking pause" when the pilots screen is first loaded due to OCR of the skill table?

I think programming an algorithm which mirrors what some of the more advanced micro-managers do, would be extremely difficult.

Good guesses! I am using PIL for screen capture but I am using pywinauto for the mouse clicking. For the OCR I am using CV2 (open CV). As you say, the OCR stuff is challenging. I don't actually have a way of reading text in the traditional sense. Rather, I have a database of images that I cross-reference when reading the game in. For example, I have individual images of the digits 0-9 in red, green, and white as they are found in the game. I have hundred of hundreds of these.

Honestly in the long run I think creating an algorithm for advanced micro-managing is a difficult challenge but not insurmountable. What is more daunting to me are the compatibility issues of getting it work for other people. And much of this is due to my OCR techniques I mentioned above; if the images I have collected from my game don't match what people are seeing exactly then the whole program may simply not work for other people.

ORIGINAL: Bullwinkle58

ORIGINAL: Sangeli

Yes, I have considered using the save file to quickly build a database for pilots. In fact that is currently my plan.

You could contact Floyd and Damien directly, but I think they had to go to Matrix/Henderson (?) and do a partial NDA and all those hoops to get access to the save file format. I think it's encrypted? Matrix has always treated the save file as the crown jewels of security. If people can play with it the PBEM portion of their marketing crashes and burns. In a game that takes real-time years to play even a hint that somebody could "edit" the save game file would be a disaster.

Good to know. I will think about that going forward.

ORIGINAL: bartrat

You could use TRACKER data instead of contacting the programmers, you can read the JDBC file I have done it using Crystal reports.

But does TRACKER dump everything? Does it dump pilot data? If so that will suffice.

ORIGINAL: kbfchicago
I would also be hesitant to trust any automated assignment tool (the game already has one I don't use, because I've ended up with transport pilots in fighter groups...) that did not have a "here is the list of what we are going to send to unit x, you good with this?" So if you continue to refine on your idea of finding max qualified & xfering via a script, I'd recommend you group the assignments, then let the player review to confirm.

IF (note caps...) you were to conclude most players leverage the Reserve Pool as a transit point (or would do so with the use of this tool). You may want to just focus on pulling lists of best qualified (by multiple skills) from the Reserve (vs. looking across all squadrons). Then allow players to pick (or at least remove) from the filtered list and execute a transfer of multiple pilots with one click. That would focus your programing (on reserve pool), provide players a means to sort on multiple skills (something not currently in the game), and potentially save a few clicks.

Thanks again, am always impressed by the (many many hours of) work by a few to develop tools that potentially help us all!

Kevin

Unfortunately I don't see the pilot tool as being as deterministic as it would require to know where pilots are going to go. The squadron commanders kind of throw a wrench in this. For example, sometimes when you draw newbie pilots into a squadron it draws in a new commander as well. Reading the pilot pools can get a good idea of pilot supply + demand but it would be unable to determine exactly where a pilot would end up. The only way to deterministically do this would be to utilize pilot selection via explicit selection (aka clicking on the pilot's name) but then of course doing that means pilot won't arrive for up to a week in transit. If you have to wait a week for pilots to arrive each time you use the tool then its not very useful at all.

[Kull]

Anybody who takes the time to make a tool that helps others (no matter how few) is deserving of kudos and thanks - so you get both! The trickiest thing for me is that you can't dump pilots into a common pool if they can be pulled by both 1E and 2E airframes, since a 1E trained pilot loses experience points (which are the slowest of all to accumulate) when moved to a 2E airframe, and vice versa (Patrol is even worse, since then you have 4E in the mix). So that means I never use common pools, which increases the mouse clicking exponentially.

Any tool that solved THAT problem would be a huge win!

[bartrat]

Tracker DOES dump a lot of pilot data. I use it manage my pilots and create Crystal reports. You can easily read the database data it uses WITHOUT having to get NDA for the save data from Matrix.
Please look in to it.

[richlove]

bartrat, are you able to read individual skill levels per pilot via Crystal? For example, can you see the NavT skill of pilot XYZ?

I spent a few minutes looking at the schema defined in tracker / trackerdb.script file; there's a "PILOTS" table but I don't see any columns that call out skill levels (although I'm not sure how far behind I am in tracker versions).

Side note: it took me a while to figure out what "CREATE CACHED TABLE TOES" was referring to [:D]

[bartrat]

It gives only EXP, not skills broken out. So would not work for you.

[geofflambert]

I don't let software run anything if I can help it. I don't even use automatic convoys. I have very complicated strategies for training, and it's different for the allies than it is for the Japanese. In my experience it's by far the most important task you have. When I was playing the Allied side I still could do 3 to 4 turns a day. If you don't do training properly against an opponent that does, plan on losing.

[Sangeli]

ORIGINAL: Kull
Anybody who takes the time to make a tool that helps others (no matter how few) is deserving of kudos and thanks - so you get both! The trickiest thing for me is that you can't dump pilots into a common pool if they can be pulled by both 1E and 2E airframes, since a 1E trained pilot loses experience points (which are the slowest of all to accumulate) when moved to a 2E airframe, and vice versa (Patrol is even worse, since then you have 4E in the mix). So that means I never use common pools, which increases the mouse clicking exponentially.

Thanks! But I haven't noticed any experience drop when switching from 1E to 2E or whatever. I've only noticed it when I go from one pool to another. Can anyone else comment on this?

ORIGINAL: bartrat
It gives only EXP, not skills broken out. So would not work for you.

Yup, so I feared. Maybe I can convince the folks behind Tracker to dump this data too. I figure if they can get experience then they can get skills fairly easily as well. Or I can get an NDA. If I can show that my automation tool is serious then maybe I can pull this off. I just need more time to invest in this. With only 1-2 hours a week this is going to take years to get a real program that I can think about giving to others. Then again, my current PBEM game may take longer to complete.

ORIGINAL: geofflambert
I don't let software run anything if I can help it. I don't even use automatic convoys. I have very complicated strategies for training, and it's different for the allies than it is for the Japanese. In my experience it's by far the most important task you have. When I was playing the Allied side I still could do 3 to 4 turns a day. If you don't do training properly against an opponent that does, plan on losing.

I'd be interested in your complicated strategies for the Allies; I'm not going to worry about the Japanese pilots since they require much less management. I'm still trying to figure out optimal algorithms.

[witpqs]

AFAIK the experience penalty is when changing to a plane that has pilots of a different pool category, as you say.

[rustysi]

Thanks! But I haven't noticed any experience drop when switching from 1E to 2E or whatever. I've only noticed it when I go from one pool to another. Can anyone else comment on this?

Yep, put a pilot trained in a single engine bomber into a twin engine and some experience is lost. Vice versa too. Makes sense when you think about it, I just wish there was a way in the pools to tell the difference.

[Sangeli]

ORIGINAL: rustysi

Thanks! But I haven't noticed any experience drop when switching from 1E to 2E or whatever. I've only noticed it when I go from one pool to another. Can anyone else comment on this?

Yep, put a pilot trained in a single engine bomber into a twin engine and some experience is lost. Vice versa too. Makes sense when you think about it, I just wish there was a way in the pools to tell the difference.

I'm not denying that it makes sense IRL. Just a game mechanic question. Can you show me an example of this with a screenshot before and after? I want to see the drop of experience on a particular pilot. Frankly, I don't know how the game would even store this info. I don't believe the game records how many engines were in the plane the pilot was training in. But I've been wrong before.

[geofflambert]

ORIGINAL: Sangeli

ORIGINAL: Kull
Anybody who takes the time to make a tool that helps others (no matter how few) is deserving of kudos and thanks - so you get both! The trickiest thing for me is that you can't dump pilots into a common pool if they can be pulled by both 1E and 2E airframes, since a 1E trained pilot loses experience points (which are the slowest of all to accumulate) when moved to a 2E airframe, and vice versa (Patrol is even worse, since then you have 4E in the mix). So that means I never use common pools, which increases the mouse clicking exponentially.

Thanks! But I haven't noticed any experience drop when switching from 1E to 2E or whatever. I've only noticed it when I go from one pool to another. Can anyone else comment on this?

ORIGINAL: bartrat
It gives only EXP, not skills broken out. So would not work for you.

Yup, so I feared. Maybe I can convince the folks behind Tracker to dump this data too. I figure if they can get experience then they can get skills fairly easily as well. Or I can get an NDA. If I can show that my automation tool is serious then maybe I can pull this off. I just need more time to invest in this. With only 1-2 hours a week this is going to take years to get a real program that I can think about giving to others. Then again, my current PBEM game may take longer to complete.

ORIGINAL: geofflambert
I don't let software run anything if I can help it. I don't even use automatic convoys. I have very complicated strategies for training, and it's different for the allies than it is for the Japanese. In my experience it's by far the most important task you have. When I was playing the Allied side I still could do 3 to 4 turns a day. If you don't do training properly against an opponent that does, plan on losing.

I'd be interested in your complicated strategies for the Allies; I'm not going to worry about the Japanese pilots since they require much less management. I'm still trying to figure out optimal algorithms.

My strategies for the Allied side are not particularly complicated in part because the US has a surplus of everything and the Brits are darn near hopeless. It's how to manage the Japanese airforces that's complicated. They have a surplus of nothing.