witpDecoder 0.5 released

[Woos]

Uff, back from Christmas holidays and that many problems. Let's see:

* The "Numeric value out of range" problem (Thanks to Lokal Yokel for the csv files) aka CHS 157/158 problem

This is caused by the "P-61A Black Widow" (Aircraft number 177) having a durability of 129 where I only provided a byte for durability (in stock nothing comes near 128). Short term solution: Edit the WITPair.csv file in a spreadsheet program (Excel or OOcalc) and set the durability of that plane to 127. Afterwards _remove_ witpload.exe from the scenario directory (or rename it). Start WitpDecoder, Initialize Database, Ignore the warning on missing witpload.exe and voila (except of course that the plane will show with a durability of 127 within witpDecoder). Don't forget to bring witpload.exe back into the scenario directory for future DB initializations of other scenarios.

Long term solution: Change the DB schema. Before I do this, I would be interested in feedback from someone in CHS who defined the numbers whether that value is really as intended. The durability data in the .csv file I got really looks skewed: The Black Widow is closely followed by 3 Beaufighters and a Ventura. The B29 is on place 23 with about 15% less durability than the Black Widow.

* Eskil's problem with scenario 5 & baskreuger's problem with scenario 2
It's a new variant of the "We make the scenario by removing some LCUs without caring whether they are HQs or something". The next minor update will be able to handle this for air groups, too. But Scenarios 2 and 5 does have production turned off anyway, why do you need witpdecoder?

* Martti got a PM. It might be another instance of "HsqlDB has problems with DELETE-INSERT sequence" which I thiught I managed to remove so be careful when using "Load Batch".

* Roger Neilson's problem with not loading the Plugin
Anyone else seeing the ClassNotFound message Roger showed in Post 22?


* US America & VSWG: No "automatically appearing" Resources/Oil in the Base overview in the Storage Tab
Well, actually that overview was never intended to show the automatically appearing things. I use them mainly to see where to expand factories and how much/fast expansion I can calculate with in the next days. Do you really _need_ it? Because it would make the code a lot more complicated.

* Screwball's "Does anyone know what this means?"
Aeh, no. Never seen that error. Are you sure it is the first one you got? You might want to delete all the witp.* files in witpdecoders directory. Reinitialize the DB and reload the savegames. Make sure you have you password in witpdecoder.bat before loading the savegame. Don't load savegames older than the latest one you loaded. But actually I have no idea what happened.

* RHS support
Not planned as I fear that too much hacking along the boundaries of the engine went into that mod. Why do you need Ship update cycles anyway (I had to agree some time ago that it is necessary for switching Petes between Fighter and Float Plane so for ACs upgrade cycles are now supported (made reading them in much slower than before)).

[Roger Neilson II]

Are there any files I can send to help to get this sorted Woos? I have your excellent utility running on two other games no problem so its got to be something related to CHS 157.

Roger

[Woos]

No I got all the files I needed from Lokal Yokel (See first bullet in post 41 for a short term fix). Your plugin problem (the first Exception meesage you get) is unrelated from this problem (which shows in the second exception message). The plugin message is a warning only anyway so it is not the cause of your main problem. The cause of that is mentioned above (I renamed the bullet heading to also include "CHS 157/158" to prevent more confusion).

[Mistmatz]

ORIGINAL: Woos
...
* RHS support
Not planned as I fear that too much hacking along the boundaries of the engine went into that mod. Why do you need Ship update cycles anyway (I had to agree some time ago that it is necessary for switching Petes between Fighter and Float Plane so for ACs upgrade cycles are now supported (made reading them in much slower than before)).

Woos, I understand that supporting RHS is much to ask for, especially with all the time and effort you spend for the upcoming AE. Thanks for that btw.

Nevertheless maybe there is a way for us users to tweak the cvs files and eg simply remove the ship class(es) in question and still benefit from your great work.
The error message I get says classes that weren't read in 800. Does that mean that class number 800 is the problem or that 800 classes are having a problem? In the latter case its probably pointless to try fixing things...

[Woos]

The error message means that there is very probably a cycle in the upgrade path of ship classes and ship class number 800 is involved in it (It is a bit strange that only 1 number is mentioned as normally at least 2 are needed for a cycle). If you look in the editor (or the WITPcls.csv file) for the "UpgradeToUnitID" field, you might be able to spot the cycle and break it in the .csv file (and then proceed similarly as described in post 41 for the "value out of range" problem). Note that changing the upgrade relationships will have serious effects on the correctness of the information shown in the "Ship Updates" and the "Ship classes" tabs.

And I fear that even if you solve that problem by .csv file editing, more will come up after it in other tables.

[Roger Neilson II]

Nope, have edited the spreadsheet as suggested, it does now come up with warning about being unable to find witpload... proceeds but then will not work as before.

Something very odd here.

That's on a completely fresh install of the utility, with the password put in - I'm quite used to all this now!

Roger

[skrewball]

Hey there Woos...thanks for the reply! I did reinstall it and it worked for my Japan game. I haven't tried for the Allies yet.

Is there a way to track individual Ship Repair?

[Local Yokel]

ORIGINAL: Woos

* The "Numeric value out of range" problem (Thanks to Lokal Yokel for the csv files) aka CHS 157/158 problem

This is caused by the "P-61A Black Widow" (Aircraft number 177) having a durability of 129 where I only provided a byte for durability (in stock nothing comes near 128). Short term solution: Edit the WITPair.csv file in a spreadsheet program (Excel or OOcalc) and set the durability of that plane to 127. Afterwards _remove_ witpload.exe from the scenario directory (or rename it). Start WitpDecoder, Initialize Database, Ignore the warning on missing witpload.exe and voila (except of course that the plane will show with a durability of 127 within witpDecoder). Don't forget to bring witpload.exe back into the scenario directory for future DB initializations of other scenarios.

ORIGINAL: Roger Neilson II

Nope, have edited the spreadsheet as suggested, it does now come up with warning about being unable to find witpload... proceeds but then will not work as before.

Something very odd here.

That's on a completely fresh install of the utility, with the password put in - I'm quite used to all this now!

Roger

Think Roger and I may have hit the same, or a similar, problem. I edited the witpair.csv file as directed, then tried initialising the database with witpload.exe disabled (renamed). That produced the following error in witpdecoder.log:

'Could not create or initialize the databasejava.sql.SQLException: bad TEXT table source file - line number: 76 Attempt to insert null into a non-nullable column: column: NAME table: WITPAIRCRAFT in statement [SET TABLE WITPAIRCRAFT SOURCE "WITPair.csv;ignore_first=true"]
java.sql.SQLException: bad TEXT table source file - line number: 76 Attempt to insert null into a non-nullable column: column: NAME table: WITPAIRCRAFT in statement [SET TABLE WITPAIRCRAFT SOURCE "WITPair.csv;ignore_first=true"]'

Examination of the witpair.csv in Excel revealed a number of empty cells in the aircraft names column (B) - the first at line 76, as the log indicated. 'Aha', thought I, 'there be the "nulls" that the schema doesn't like' - so I substituted 'DUMMY' for each occurrence of such an empty cell in the aircraft names column of the witpair.csv file, and tried to re-initialise the database. That cured the problem [8D], once I remembered to edit witpdecoder.bat by inclusion of my password [:o].

Roger, this may be the solution to your problem - HTH

Woos, this enhancement looks excellent - many thanks for identifying the source of my problem and the solution to it [&o][&o][&o]

[Andrew Brown]

ORIGINAL: Woos
* The "Numeric value out of range" problem (Thanks to Lokal Yokel for the csv files) aka CHS 157/158 problem

This is caused by the "P-61A Black Widow" (Aircraft number 177) having a durability of 129 where I only provided a byte for durability (in stock nothing comes near 128). Short term solution: Edit the WITPair.csv file in a spreadsheet program (Excel or OOcalc) and set the durability of that plane to 127. Afterwards _remove_ witpload.exe from the scenario directory (or rename it). Start WitpDecoder, Initialize Database, Ignore the warning on missing witpload.exe and voila (except of course that the plane will show with a durability of 127 within witpDecoder). Don't forget to bring witpload.exe back into the scenario directory for future DB initializations of other scenarios.

Long term solution: Change the DB schema. Before I do this, I would be interested in feedback from someone in CHS who defined the numbers whether that value is really as intended. The durability data in the .csv file I got really looks skewed: The Black Widow is closely followed by 3 Beaufighters and a Ventura. The B29 is on place 23 with about 15% less durability than the Black Widow.

What version of CHS was this from? I just checked the latest version of scen 157 from my website (2.08c) and it has a value of 48 for the P-61A durability.

Andrew

[Roger Neilson II]

Well I tried that, I'm actually on 158 not 157 - but the empty slots were there so I filled them in as suggested. I also corrected the Black Widow as per Andrew's post and previously Woos' post.

Fresh install (again) fresh attempt at initialising produces the following log file:
java.lang.ClassNotFoundException: de.retsiemuab.witpDecoder.gui.mapviewPlugins.TestPlugin
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at de.retsiemuab.witpDecoder.gui.am.<init>(Unknown Source)
at de.retsiemuab.witpDecoder.gui.H.a(Unknown Source)
at de.retsiemuab.witpDecoder.Main.a(Unknown Source)
at de.retsiemuab.witpDecoder.Main.main(Unknown Source)
Not loading one/all plugins.
Could not create or initialize the databasejava.sql.SQLException: Numeric value out of range
java.sql.SQLException: Numeric value out of range
at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.jdbcStatement.executeUpdate(Unknown Source)
at de.retsiemuab.witpDecoder.d.a(Unknown Source)
at de.retsiemuab.witpDecoder.ioreaders.c.a(Unknown Source)
at de.retsiemuab.witpDecoder.d.b(Unknown Source)
at de.retsiemuab.witpDecoder.gui.at.run(Unknown Source)
at org.eclipse.swt.custom.BusyIndicator.showWhile(Unknown Source)
at de.retsiemuab.witpDecoder.gui.u.d(Unknown Source)
at de.retsiemuab.witpDecoder.gui.i.widgetSelected(Unknown Source)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at de.retsiemuab.witpDecoder.gui.H.a(Unknown Source)
at de.retsiemuab.witpDecoder.Main.a(Unknown Source)


Previous logfile for comparison:

java.lang.ClassNotFoundException: de.retsiemuab.witpDecoder.gui.mapviewPlugins.TestPlugin
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at de.retsiemuab.witpDecoder.gui.am.<init>(Unknown Source)
at de.retsiemuab.witpDecoder.gui.H.a(Unknown Source)
at de.retsiemuab.witpDecoder.Main.a(Unknown Source)
at de.retsiemuab.witpDecoder.Main.main(Unknown Source)
Not loading one/all plugins.
Could not create or initialize the databasejava.sql.SQLException: Numeric value out of range
java.sql.SQLException: Numeric value out of range
at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.jdbcStatement.executeUpdate(Unknown Source)
at de.retsiemuab.witpDecoder.d.a(Unknown Source)
at de.retsiemuab.witpDecoder.ioreaders.c.a(Unknown Source)
at de.retsiemuab.witpDecoder.d.b(Unknown Source)
at de.retsiemuab.witpDecoder.gui.at.run(Unknown Source)
at org.eclipse.swt.custom.BusyIndicator.showWhile(Unknown Source)
at de.retsiemuab.witpDecoder.gui.u.d(Unknown Source)
at de.retsiemuab.witpDecoder.gui.i.widgetSelected(Unknown Source)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at de.retsiemuab.witpDecoder.gui.H.a(Unknown Source)
at de.retsiemuab.witpDecoder.Main.a(Unknown Source)
at de.retsiemuab.witpDecoder.Main.main(Unknown Source)
Couldn't set up a working DB template. Bad, Bad.


They both look the same to me...... so no progress

Roger

[Woos]

@Andrew
No idea, I'm always only interested in the .csv files. Local Yokel should know.
But since the high values therefore seem to be unintentional I will not change the schema to allow more than 127 durability. Instead the initialization routine will catch this.

@Roger
Stop trying, new version which solves this is in the works.

[Roger Neilson II]

ok, thanks

Roger

[Woos]

A new version (0.5a) is available from http://extweb.retsiemuab.de/witp/witpdecoder0.5a.zip . It is the whole zip file (and not only the jar) since I also included a new version of hsqldb.jar (although that didn't help with the Load Batch problem)).

Fixed:

• CHS 157/158 problem: Now witpDecoder simply uses 127 as durability and warns you about it.
• Scenario 2 & 5: Missing HQs for Airgroups are now allowed, so the scenarios should load. Be aware that if all planes of one type have missing HQs, they will only show in the "AC classes" tab if "All nationalities" are selected.
• Load Batch might now work. I could reproduce Martti's problems with the Allies and "fixed" it by increasing the checkpoint frequency. Of course that is not really a fix but it seems to work now.
• On special request from VSWG the numbers of Ships Available/Building/Sunk are now listed per class in the "Ship classes" tab. Note that for witpdecoder each update is an own class, so to see all ship, select the "All" choice for "Availability". Otherwise you will not see ships building for future updates and (once you have updated) won't see ships of out-of-date classes anymore.

[Roger Neilson II]

[&o]

Roger

[Local Yokel]

ORIGINAL: Andrew Brown

ORIGINAL: Woos
* The "Numeric value out of range" problem (Thanks to Lokal Yokel for the csv files) aka CHS 157/158 problem

This is caused by the "P-61A Black Widow" (Aircraft number 177) having a durability of 129 where I only provided a byte for durability (in stock nothing comes near 128). Short term solution: Edit the WITPair.csv file in a spreadsheet program (Excel or OOcalc) and set the durability of that plane to 127. Afterwards _remove_ witpload.exe from the scenario directory (or rename it). Start WitpDecoder, Initialize Database, Ignore the warning on missing witpload.exe and voila (except of course that the plane will show with a durability of 127 within witpDecoder). Don't forget to bring witpload.exe back into the scenario directory for future DB initializations of other scenarios.

Long term solution: Change the DB schema. Before I do this, I would be interested in feedback from someone in CHS who defined the numbers whether that value is really as intended. The durability data in the .csv file I got really looks skewed: The Black Widow is closely followed by 3 Beaufighters and a Ventura. The B29 is on place 23 with about 15% less durability than the Black Widow.

What version of CHS was this from? I just checked the latest version of scen 157 from my website (2.08c) and it has a value of 48 for the P-61A durability.

Andrew

ORIGINAL: Woos

@Andrew
No idea, I'm always only interested in the .csv files. Local Yokel should know.
But since the high values therefore seem to be unintentional I will not change the schema to allow more than 127 durability. Instead the initialization routine will catch this.

Andrew, I can tell you that the CHS scenario 157 version from which I derived the files sent to Woos was 2.08, but I can't tell you whether it was 2.08c or some other sub-version. "CHS 2.08 + Nik mod" is all that appears in the scenario selection screen within the game. Is there a way in which I can distinguish one sub-version from another?

[RAM]

errrrrrrrr...more problems in this end, when trying to load a savegame (now yes, it's when trying to load a savegame [:'(])

Couldn't write things to the DB due to java.sql.SQLException: Integrity constraint violation - no parent AGsBaseMustExist table: BaseStatus
java.sql.SQLException: Integrity constraint violation - no parent AGsBaseMustExist table: BaseStatus
at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.jdbcStatement.executeUpdate(Unknown Source)
at de.retsiemuab.witpDecoder.d.a(Unknown Source)
at de.retsiemuab.witpDecoder.data.c.e(Unknown Source)
at de.retsiemuab.witpDecoder.data.c.a(Unknown Source)
at de.retsiemuab.witpDecoder.data.b.b(Unknown Source)
at de.retsiemuab.witpDecoder.b.a(Unknown Source)
at de.retsiemuab.witpDecoder.gui.aT.run(Unknown Source)
at org.eclipse.swt.custom.BusyIndicator.showWhile(Unknown Source)
at de.retsiemuab.witpDecoder.gui.u.a(Unknown Source)
at de.retsiemuab.witpDecoder.gui.b.widgetSelected(Unknown Source)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at de.retsiemuab.witpDecoder.gui.H.a(Unknown Source)
at de.retsiemuab.witpDecoder.Main.a(Unknown Source)
at de.retsiemuab.witpDecoder.Main.main(Unknown Source)



[&:][&:][&:]

[Andrew Brown]

ORIGINAL: Local Yokel
Andrew, I can tell you that the CHS scenario 157 version from which I derived the files sent to Woos was 2.08, but I can't tell you whether it was 2.08c or some other sub-version. "CHS 2.08 + Nik mod" is all that appears in the scenario selection screen within the game. Is there a way in which I can distinguish one sub-version from another?

I checked the file contents and no there isn't. Something I will have to fix. But in the meantime the safest thing to do would be to download a new copy of the scenario file.

Andrew

[Eskil]

ORIGINAL: Woos

[snip]

* Eskil's problem with scenario 5 & baskreuger's problem with scenario 2
It's a new variant of the "We make the scenario by removing some LCUs without caring whether they are HQs or something". The next minor update will be able to handle this for air groups, too. But Scenarios 2 and 5 does have production turned off anyway, why do you need witpdecoder?

[snip]

I am currently trying to get the hang of the game and figured I might as well get used to the unofficial tools in the process as well. Apart from that I am using the tab with future objectives of LCUs so there is actually a point with your utility. It is working now as well apart from tab with historical ship movement but that is most likely due to user error. This information is only available once you have loaded multiple save files, right? If this is the case, what's a good strategy to handle to the save slots? I read somewhere that you get get auto saves weekly and daily, are these overwritten or renamed so you can use those?

/Eskil

[Nomad]

ORIGINAL: Woos

The error message means that there is very probably a cycle in the upgrade path of ship classes and ship class number 800 is involved in it (It is a bit strange that only 1 number is mentioned as normally at least 2 are needed for a cycle). If you look in the editor (or the WITPcls.csv file) for the "UpgradeToUnitID" field, you might be able to spot the cycle and break it in the .csv file (and then proceed similarly as described in post 41 for the "value out of range" problem). Note that changing the upgrade relationships will have serious effects on the correctness of the information shown in the "Ship Updates" and the "Ship classes" tabs.

And I fear that even if you solve that problem by .csv file editing, more will come up after it in other tables.

Ship Class errors can be reported as one number. I set up Gary Childress's Mod and it had three ship classes that had upgrades to blank entries. the blank ones did not get reported in the error report. BTW, Garys mod will work by setting those three to upgrade to themselves.

[Mistmatz]

ORIGINAL: Nomad

ORIGINAL: Woos

The error message means that there is very probably a cycle in the upgrade path of ship classes and ship class number 800 is involved in it (It is a bit strange that only 1 number is mentioned as normally at least 2 are needed for a cycle). If you look in the editor (or the WITPcls.csv file) for the "UpgradeToUnitID" field, you might be able to spot the cycle and break it in the .csv file (and then proceed similarly as described in post 41 for the "value out of range" problem). Note that changing the upgrade relationships will have serious effects on the correctness of the information shown in the "Ship Updates" and the "Ship classes" tabs.

And I fear that even if you solve that problem by .csv file editing, more will come up after it in other tables.

Ship Class errors can be reported as one number. I set up Gary Childress's Mod and it had three ship classes that had upgrades to blank entries. the blank ones did not get reported in the error report. BTW, Garys mod will work by setting those three to upgrade to themselves.

Good hint Nomad, I'll try that once I have some spare time. Thanks.