Intel Monkey 1.0 Pre-Beta

[witpqs]

Calling this Pre-Beta but it's had a heck of a workout and lots of development. Have a look.

https://sites.google.com/view/staffmonkeys/home

Special thanks to forum members Roger Neilson aka Roger Neilson 3 (formerly Roger Neilson and Roger Neilson 2) and Walter Eardley aka Andav (better known to my AAR readers as the Devious Admiral Wa) They have provided assistance of all kinds for the past 4 1/2 months. Their advice, testing, and general able assistance has made Intel Monkey gel into a smooth application.



Edit to post new link: https://sites.google.com/view/staffmonkeys/home

[Bullwinkle58]

ORIGINAL: witpqs

Calling this Pre-Beta but it's had a heck of a workout and lots of development. Have a look.

https://sites.google.com/site/staffmonkeys/intel-monkey

Let me give a big +1 to this utility. I've been using the pre-pre-beta for years, and it works great. It does things the other utilities don't. For the Allies especially, pre-invasion looks at islands and what has been reported there over the year(s) is a great help. It sounds like witpqs has made this install even easier by incorporating Python into the installer.

I don't use Monkey every turn, or even every game month. But when you need what it can do you'll be thankful you have it.

[BillBrown]

I installed the new version. I had it process new files and it looks like all I got was the sigint files. There is a memory error report in one of the errfile logs
Process CR:
Traceback (most recent call last):
File "multiprocessing\process.py", line 258, in _bootstrap
File "multiprocessing\process.py", line 93, in run
File "lib\IP_procs.py", line 58, in do_combatreport
File "lib\IP_combat_rpt.py", line 510, in __call__
MemoryError

I do have almost 2 years of reports stored for this game. Do I need to crop the reports in the archive folder?
I do have 16 GB of RAM installed, but I think only 2 GB are available in a 32-bit environment.

These are the last few lines in the intel monkey window:

Finished moving data from CSV files to the database.
Get Scenario Data operation completed.
Load Info operation beginning.
Load Bases Info Completed.
Load LCUs Info Completed.
Load Air Groups Info Completed.
Load Aircrafts Info Completed.
Load Ships Info Completed.
Load Info Summary:
Bases Info Loaded.
Lcu_types Info Loaded.
Groups Info Loaded.
Aircraft Info Loaded.
Ships Info Loaded.
Load Info operation completed.
Begin Processing Reports.
SigInt Reports Processed Statistics
FirstDay : 1941-12-07
LastDay : 1943-09-23
DaysProcessed : 656
FilesProcessed : 656
RecordsProcessed : 19,393
Process SigInt Reports Elapsed Time 0:00:01.840804
Finished processing SigInt files.
Error processing AE Combat reports. Consult log files.
Operations files not implemented.
Finished processing Operations files.
Events files not implemented.
Finished processing Events files.
Finished Processing Reports.

[witpqs]

ORIGINAL: BillBrown

I installed the new version. I had it process new files and it looks like all I got was the sigint files. There is a memory error report in one of the errfile logs
Process CR:
Traceback (most recent call last):
File "multiprocessing\process.py", line 258, in _bootstrap
File "multiprocessing\process.py", line 93, in run
File "lib\IP_procs.py", line 58, in do_combatreport
File "lib\IP_combat_rpt.py", line 510, in __call__
MemoryError

I do have almost 2 years of reports stored for this game. Do I need to crop the reports in the archive folder?
I do have 16 GB of RAM installed, but I think only 2 GB are available in a 32-bit environment.

These are the last few lines in the intel monkey window:

Finished moving data from CSV files to the database.
Get Scenario Data operation completed.
Load Info operation beginning.
Load Bases Info Completed.
Load LCUs Info Completed.
Load Air Groups Info Completed.
Load Aircrafts Info Completed.
Load Ships Info Completed.
Load Info Summary:
Bases Info Loaded.
Lcu_types Info Loaded.
Groups Info Loaded.
Aircraft Info Loaded.
Ships Info Loaded.
Load Info operation completed.
Begin Processing Reports.
SigInt Reports Processed Statistics
FirstDay : 1941-12-07
LastDay : 1943-09-23
DaysProcessed : 656
FilesProcessed : 656
RecordsProcessed : 19,393
Process SigInt Reports Elapsed Time 0:00:01.840804
Finished processing SigInt files.
Error processing AE Combat reports. Consult log files.
Operations files not implemented.
Finished processing Operations files.
Events files not implemented.
Finished processing Events files.
Finished Processing Reports.

First off, my email address is on the web page (maybe mouse-over my name there). Otherwise PM me and I will give it to you (to avoid spambots picking up email addresses from the forum).

No need to crop the reports. No worry about memory. A separate process is started to process SigInt reports and one to process Combat reports. Because catching up many turns of Combat reports is a huge amount of volume, that process writes records to the database whenever it has 1,000 of any type. It actually uses less memory than most of the reports use.

What almost certainly happened is that it encountered something in a Combat report that it could not handle and barfed on it. I need the logs from that run in the debug folder. When you start Intel Monkey there is a group of files with a similar name under _MAIN, then when the processes start to read in AE reports other groups are created _SI and _CR for SigInt and Combat, respectively. I need the group for _CR.

[witpqs]

Adding a general note to the reply to Bill Brown:

Loads of unusual, rare, and even erroneous conditions in Combat reports have been found in testing and accommodated. Many conditions that have been fixed by the AE developers are still accommodated for so that old reports can be read in. There must be more lurking out there, and with examples I can make changes to accommodate them.

[BillBrown]

I only have one file with CR in the name and it only has one line in it. I will send it to you but here it is

Starting at 2017:11:15:13:42:35:986850

[witpqs]

ORIGINAL: BillBrown

I only have one file with CR in the name and it only has one line in it. I will send it to you but here it is

Starting at 2017:11:15:13:42:35:986850

I should have spelled this out better. The file with the abbreviation is a 'marker' to show which groups of logs is which. Notice that the log file names have a date-time stamp in their names so that they sort together in your folder display. I need all the files from the CR group.

[BillBrown]

OK, tonight then.

[witpqs]

Special thanks to forum members Roger Neilson aka Roger Neilson 3 (formerly Roger Neilson and Roger Neilson 2) and Walter Eardley aka Andav (better known to my AAR readers as the Devious Admiral Wa) They have provided assistance of all kinds for the past 4 1/2 months. Their advice, testing, and general able assistance has made Intel Monkey gel into a smooth application.

I updated the first post with the above acknowledgements but I realize that most people will not see the old post again, so I am repeating it here.

Also, I have put a troubleshooting page on the web site with information about the log files.

[witpqs]

ORIGINAL: BillBrown

I installed the new version. I had it process new files and it looks like all I got was the sigint files. There is a memory error report in one of the errfile logs
Process CR:
Traceback (most recent call last):
File "multiprocessing\process.py", line 258, in _bootstrap
File "multiprocessing\process.py", line 93, in run
File "lib\IP_procs.py", line 58, in do_combatreport
File "lib\IP_combat_rpt.py", line 510, in __call__
MemoryError

I do have almost 2 years of reports stored for this game. Do I need to crop the reports in the archive folder?
I do have 16 GB of RAM installed, but I think only 2 GB are available in a 32-bit environment.

These are the last few lines in the intel monkey window:

Finished moving data from CSV files to the database.
Get Scenario Data operation completed.
Load Info operation beginning.
Load Bases Info Completed.
Load LCUs Info Completed.
Load Air Groups Info Completed.
Load Aircrafts Info Completed.
Load Ships Info Completed.
Load Info Summary:
Bases Info Loaded.
Lcu_types Info Loaded.
Groups Info Loaded.
Aircraft Info Loaded.
Ships Info Loaded.
Load Info operation completed.
Begin Processing Reports.
SigInt Reports Processed Statistics
FirstDay : 1941-12-07
LastDay : 1943-09-23
DaysProcessed : 656
FilesProcessed : 656
RecordsProcessed : 19,393
Process SigInt Reports Elapsed Time 0:00:01.840804
Finished processing SigInt files.
Error processing AE Combat reports. Consult log files.
Operations files not implemented.
Finished processing Operations files.
Events files not implemented.
Finished processing Events files.
Finished Processing Reports.

Fixed.
Processing Combat reports processing was not properly handling a day with no combat.

New version posted to the web site.
New version sha256 hash is:
1240cac98737166d5df4813269c8584101a1f8e4b97b59cdb95b8f3f80e28fb2

[witpqs]

Noticed that the installer was not writing over old files. Fixed. Posted on the web site.

sha256 hash for IntelMonkeyInstaller.exe
6cc8b5bc9b39a112169667d622ecb967d3c29c99eb0d695ffed48c338b7c7fe5

[Lowpe]

Christmas comes early![&o]

[witpqs]

For those who want to check, I have been including the sha256 hash values of the IntelMonkey.exe file and the IntelMonkeyInstaller.exe file. I've now posted a custom utility to calculate the hash values of those files on your system. It looks for those files in the folder where you run it and if it finds them displays their hash values.

https://sites.google.com/site/staffmonkeys/intel-monkey/intelmonkeyhash-utility

This can be useful if you want to see if you have the same version of IntelMonkey.exe and IntelMonkeyInstaller.exe as the ones on the web site. During these early months of development there might be many updates to Intel Monkey.

[RogerJNeilson]

Got to say I am very surprised by the low level of response to this, You guys need to try it.... I could go on at length but won't at this point.

Roger

[witpqs]

A little appetite whetting...

Above you've seen the log files for troubleshooting. A startup pictorial will come later. Now for a little bit about normal operations.

Here is the main menu. Notice the report buttons in the top area and the more administrative buttons in the lower area. The large box for a message area let's you know what is going on.

On the right of the header area you see "Test Game". That's user-selectable text to help you keep things straight when you have multiple games going and have more than one Intel Monkey open.

[witpqs]

If you vertically stretch the window, the message area takes all of the additional space.

[witpqs]

The message area scrolls in case you want to review something.

[witpqs]

The configuration menu. These are user configuration items. When you configure "AE Home" doing so will also automatically configure the items in the same group. If you are doing testing like I was when I took this pic, you might then set "archive" to point somewhere else, but normally you would just leave it pointing to "SAVE/archive".

[witpqs]

Database maintenance. If and when you want to create a new database all you have to do is push a button. The old database will be renamed (-old##) in case you change you mind or made a mistake.

As Intel Monkey is further developed new features will be added, and some of those will require new database tables and fields. After an Intel Monkey upgrade the databases can be brought up to the new specs with a click.

[witpqs]

The heart of Intel Monkey is the reports. Everything else exists to bring you the reports. Here is what the SigInt report look like when you first start it.