Coder Diary #20 -- The 2.01 Bug-Fix Update

[berto]

Coder Diary #20 -- The 2.01 Bug-Fix Update


The 2.01 Bug-Fix Update -- So what's taking so long? Well ... bug fixing!

Especially reviewing, fixing, and testing DCGs (Dynamic Campaign Games), Random Battle generation, and all the many other "frontend" functions and game features.

"frontend" refers to the program for launching new games, campaigns, and random battles, also the editors; the program depicted here:



Little-known fact: John Tiller did not author the JTCS frontend!

The JTCS frontend was a separate project authored by one or more programmers at Talonsoft, the original Campaign Series publisher. Different in so many ways -- organization, layout, coding style, etc. -- from the other programs in the series (edmap, edit, edorg, and the game engine), the frontend has languished over the years, with few if any updates, and more than a few unfixed bugs. This was especially so as new games, and many new nations, were added to the mix.

So much of our attention over the past couple of months has been devoted to fixing frontend issues.

Also DCGs. You already know about the DCG PlayVictory() crash bug, since fixed. That was just the first among many DCG/random battle bugs we found. It's been a slow and painful process, finding and fixing the bugs, then testing the fixes. By their very nature, Dynamic Campaign tests can't be rushed (and unfortunately can't be automated). Each campaign test necessitates playing two or more scenarios. That takes time, quite a bit of it.

Here is the latest Change Log for the 2.01 Update:

General Improvements

[*]Implemented Options > Toolbar > None/Classic/Standard/Large.
[*]Implemented Options > System Colors.
[*]Implemented Help > Unit Viewer (F3).
[*]Implemented Help > Manual (F1) in edit, edmap & edorg; also Help > Unit Viewer (F3) in edit.
[*]Implemented the -X NoEncryption option implemented in all of: engine, edit, edorg, edmap.
[*]Implemented a moddable timelines.dat, for customizing nation timelines.
[*]Corrected several nation timelines.
[*]Added a new Campaign.pdt and .pdx data file for all engines.
[*]Added a new Main.pdt and .pdx data file for all engines (supply covers now all countries for 1930-1952).
[*]Added, in ai.ini and elsewhere, three new moddable A/I parameters: AI_smoke_prob, AI_blind_fire_prob, AI_dig_prob.
[*]Tweaked the highlight/selected colors for greater color contrast.
[*]The game engine now defaults to window maximized mode on startup.
[*]Adapted paradrop and airplane animations to the new Variable Animation Speed system. (Also, flight animations are now capped at 80% of the max ground unit animation speed.)
[*]Added more logreturn() calls and better exception handling for missing files.
[*]Improved frontend exception handling and crash prevention.
[*]Improved and extended frontend error reporting/logging.
[*]Eliminated many code & data redundancies between the (TS) frontend and the other four (JT) EXEs.

Graphics Improvements (for now, for West Front only; East Front and Rising Sun to follow in Patch 2.10)

[*]Most units now will have a unique 2D picture on the game counter when using graphical icons.
[*]Units were all given a military icon to add more variety to the very limited original mix. The original game had less than 90, now there are 303!
[*]Enhanced many of the 3D graphics as well.
[*]Updated all countries to include common files such as trains, supply depots, etc.
[*]Updated a lot of the Minor countries kit, especially Belgium, France, Italy, Albania, Yugoslavia, Norway, and Ethiopia.

Bug Fixes

[*]Fixed the A/I "dithering unit" bug, and implemented A/I > Prohibit A/I Backtracking.
[*]Fixed the campaign game PlayVictory() crash bug, and various other DCG-related crash bugs.
[*]Fixed various issues with Campaign.pdt, and recoded much of how frontend processes Campaign.pdt data tables.
[*]Fixed several other frontend bugs.
[*]Fixed the F2 Unit Handbook unit text bug.
[*]Fixed the broken frontend scenario list sort-by-date.
[*]Fixed several broken options settings saves/restores.
[*]Fixed the broken Options > Opaque Infobox check (selected) display.
[*]Applied various other minor bug fixes and code improvements.
[*]Added a correct Icons[3/4/5]d.bmp with Russian bases properly aligned colorwise.
[*]Added new versions of Icons[1/2]d.bmp files.
[*]West Front: All stock scenarios and Linked Campaign Games (LCG) that were affected by the Unit ID change in patch 1.03 were modified to use correct units.
[*]Added missing unit Bitmap and Bit files.

The Change Log items highlighted above will be of special interest to modders. We'll have more to say about 2.01's expanded modding capabilities soon.

A screenshot showcasing some of Warhorse's recent 2D unit icon work (from a "throwaway" DCG test game, where the purpose of the game was to lose!):



Awfully pretty, that! [8D]

Much, much work has been done in the area of error logging/reporting. Some interesting new error dialogs:



Before 2.01, the frontend quite possibly might crash at the point of that kind of error.

And another:



Before 2.01, if you specified an unsupported date, the frontend would report tersely: "Selected date is out of range. Please reselect." With 2.01, the frontend now also reports the valid date range, so you no more have to guess a proper date.

Note in the screenshot the attempt to specify 4/1930 (April 1930) as the random battle date. As you can see, the supported dates are 1/1935 to 12/1946 (these have been changed from 2.00 and before). With 2.01, you have the option to extend the nation timelines by modding the new timelines.dat file. (Again, we'll have more to say about this soon.)

Not that you would know it, but: We've been quite busy striving to make JTCS 2.01 as bug-free and polished as reasonably possible. Added a few new features along the way too.

Our hard work, and your patience, will soon be rewarded: We are in the very final stages of packaging the 2.01 Update.

There's many a slip twixt the cup and the lip, but we hope to be releasing the JTCS 2.01 Update in the very near future.

We trust you'll find it all to be well worth the wait.

Until the next time...

[berto]

Oh, I'd forgot: Thanks to Jason, with the 2.01 Update we'll have an updated Manual too!

[Josh]

"... It's been a slow and painful process, finding and fixing the bugs, then testing the fixes......"

Thx for all your hard work guys! [&o]

[Jafele]

I love all the changes and fixes you´re implementing. It´s an absolutely professional work. There´s no hurry if we´re going to have a firmly stable version. I don´t mind to wait all the time required to polish this valuable gem.

Thanks to everyone involved! [&o]

[Big Ivan]

Very nice work berto and the team!

Thank you, it is appreciated.

Best Always!!

Big Ivan

[Crossroads]

Thanks Berto! Here's a few additional thoughts on the new 2.01...

Who would have thought there's such a full application behind that innocent looking Main Menu!

Not me, for one. [8|]

Here's a little Soap Box:

I've always been a PBEM player, the skirmishes I've had vs AI have been few and far between. The last time I spent significant time vs AI was when I played through all Panzer Leader cardboard game conversion scenarios a couple of years ago. Panzer Leader France 1940 situations were of particular fun!

So when I learned I need to put a thorough testing session vs Front End generated battles, I approached the job with, how should I put it, less than perfect enthusiasism...

Little did I know!

Berto's put a huge effort in, in integrating front end code into game engine code. A lot of stuff was duplicated, country codes for one, but a lot of other variables as well. Variables that until now have been hard coded to the programming code. No variation available, often no information about them available either. [:(]

For the front end testing purposes, Berto had to share some code snippets to us, for us to know what date sequences were valid, and what were not.

Of course, we spotted some inconsistencies. For an example, it was possible to have dynamic battles with Russians only as of November 1939.

Soviet invasion of Poland? Nope. Not supported.

First of all, Russia is "Allied", as is Poland. There, of course is the "Axis" Soviets, but they do not have a Campaign24.pdt with them, which is needed for these generated battles. Also, Axis Soviet timeline begun from 11 1939 too.


Battle Generator Reborn

With the current OOBs, the availability for many platoons has been extended to cover a period from 1930 to 1952. However, often there are no higher level organizations available (battalions, divisions, corps, ...) for the whole extended period. And even if there were, how would we go about testing the changes, if the code would be changed to allow for a longer time period for generated battles? Manual testing remember, twice per test case, preferably.

Axis Soviets alone: Scenario size [Bn to Corps, 5 options,], then Engagement type [Meeting engagement to Mountain Pass, 11 options ]. So, there would be 5 * 11 * 2 = 110 test cases if we'd do a perfect testing over Soviets alone. No can do... [:(]

So what to do? We left them (for most parts) as-is, improved the error handling, and: made the timelines a moddable parameter file!

And this was when the serious fun started...

Instead of testing existing countries and timelines, that have been there for ages and would work for most part, I decided to try something new. [:)] How does this sound:

East Front:

- Soviet Invasion of Poland, Sept-Nov 1939, Axis Soviets vs Allied Poles
- Poland - Slovakia Border Battles, the same time period but with Axis Slovaks vs Allied Poles
- Finnish - German Lapland war, from Sept 1944 to May 1945, with Allied Finns vs Axis Germans,
- an uprising with Allied Germans vs Axis Germans, May 1945?

Or, West Front:

- Axis Soviets vs US?

Or, Rising Sun:

- Japanese Invasion of Manchuria, Sept 1931 to Feb 1932, with Axis Japan vs Allied Nat. China
- Japanese Dutch East Indies Campaign, from Dec 1941 to March 1942, or
- Soviet-Japanese War in Manchukuo, in August 1945.

All needed was to modify the appropriate timelines.dat, and to add a Campaign##.pdt if one was missing. Or add organizatios to it, if the time period was missing. Easy! [8D]

So what did we learn? First: you really do not (!) want to wage your Type 95 HA-GO tanks against the ISU-152s! SU-76s? If you have to. ISU-152s? No way!




What else? That it was tons of fun! What was not available pre-2.01, you can now mod pretty much as you want


Random Battles Like You Have Never Seen Before

And the fun does not stop here. Now you can really tweak your generated battles to the max.

For the Dutch East Indies campaign, I decided to replace the existing Campaign11.pdt with one that only has the Dutch East Indies forces, the Java Defense Force from the Corps11.oob. Now, the generated battles use only elements of this formation. How cool is that:



You can now generate say Afrika Korps random battles too:

- Create the two Campaign##.pdt files, one with Desert Rats only, and one with Afrika Corps only.
- If so preferred, modify the timelines.dat so that it supports the organization timespan there.
- For additional touch, package the respective Unit bases from the Unit Base Mod by XLVIII Pz. Korp for additional authencity:



- And, importantly, package it to be used as a user mod, with JSGME. This way the original files do not get lost and can easily be put back. More of that later.

Sweet [:)]


Editing and Further Modifying Randomly Generated Battles

Wait. There's still some more [:)] As you may know, the engine generates the scenario files to go with the generated battle.

The three files are available in the game folder with a Random##.* syntax, one each for scenario (.SCN), organization (.ORG), and map (.MAP).

The Dutch East Indies scenario I created using the RS Battle Generator was the third one I did, hence all the files will have the Random2 prefix. Here's the ORG file from the Dutch East Indies example:



And here's the Scenario Editor view on Random2.SCN file (The new Manual and Arkady's Unit Viewer available to assist, from Help menu)




You can continue to modify the generated scenario files with the various editors, and as the case often is, the result will be all better for it if you do so.

With 2.01 having the -X option (no encrypted files needed for vs-AI games) available to all editors, you can add your own units to the mix.

It is all open now [X(]


Installing and Uninstalling Your Mods: JSGME Made Easy

All this modding requires the right tools, and it is our strong recommendation not to tamper with the game file system. There is some 25 000 files there, and one little error can cause the game crashing. Trust us, we know!

Instead, do package your modifications under user mods folder, and use JSGME to enable (install) and disable (uninstall) them. 2.01 will come with even a stronger integration with JSGME mod management tool. All user mods will be packaged under one /Mods subfolder, and the JSGME.exe will be in place in a fire-and-forget mode. The user mods have been repackaged as well:



As you can see, the Black F2 Info Boxes can now be easily deployed to all games, while the other mods such as the Alternative Water remains to be available to enable per game. Using JSGME is easy as a cake, so no need to take unnecessary risks while modifying the game content.

I am really hoping to see a lot of mods out here that use this new feature. Shared fun is more fun, right! [&o]


New 2.01 Manual

This!

ORIGINAL: berto
Oh, I'd forgot: Thanks to Jason, with the 2.01 Update we'll have an updated Manual too!

Jason had to create the whole manual pretty much from scratch. Here's a big hand for him, was quite an effort let me assure you! [&o]


So Long, and Thanks for All the Fish!

Meanwhile, we as the team hope to finally be able to continue with the Modern Wars project, and to leave the Campaign Series to your Tender Loving Care

Campaign series will return in 2015, with the 2.1 content release.

Included there will be among other things Warhorse's complete rework of the EF and RS organizations together with their new 2D graphics. You can already enjoy his work with WF.

As all work makes everyone a dull boy, Warhorse and I are planning to present some previews of things to come, possibly in a format of a user mod (Forgotten Battles 2.0?).

I for one can't wait for his complete rework of the Polish OOB, to cover the whole period. Maybe something that would be worth a sneak peek, don't you think?

[Jafele]

Interesting additions, Crossroads. The future of JTCS is in the hands of modders and you made the first and more important step to get it. Obviously all of you put your hearts into it, because this is not common to see such an amazing work. [&o]

I´d like to know if you are going to make an upgrade with the Modern Wars project.

[Crossroads]

I believe the longevity of the Campaign Series very much results from the open nature of the game. The editors have been there from the beginning, also originally there were no encryption with the key files. Now, while the encryption remains in place to support play-by-email-games, the sandbox is fully open for the vs-AI modifications.

-X option was a really nice addition, and now the front end becomes a bit more open as well. I know that for an example the Afrika Korps mod could be done with existing version as well, as both UK and German have their Campaign##.pdt files, and their timelines of course support the African campaign. However I wanted to put in a shameless plug for XLVIII Pz Korp's fab unit bases! [:D]

[berto]

Before 2.01, if you input unsupported dates in the Battle Generator, you would see something like this:



Leaving you clueless as to what an in-range date might be.

Soon, with 2.01, you will instead have:

ORIGINAL: berto



Before 2.01, if you specified an unsupported date, the frontend would report tersely: "Selected date is out of range. Please reselect." With 2.01, the frontend now also reports the valid date range, so you no more have to guess a proper date.

Note in the screenshot the attempt to specify 4/1930 (April 1930) as the random battle date. As you can see, the supported dates are 1/1935 to 12/1946 (these have been changed from 2.00 and before). With 2.01, you have the option to extend the nation timelines by modding the new timelines.dat file...

Here is timelines.dat, highlighting the needed modifications to support the desired 4/1930 date (and even earlier, to 1/1930):

---

11 1939 5 1945 # XRussiaNation
9 1939 5 1945 # XGermanyNation
8 1941 3 1943 # XItalyNation
7 1941 8 1944 # XRomaniaNation
7 1941 5 1945 # XHungaryNation
11 1939 9 1944 # XFinlandNation
6 1941 6 1944 # XSlovakiaNation
9 1939 10 1939 # XPolandNation
9 1939 6 1940 # XFranceNation
9 1939 5 1945 # XBritainNation
1 1930 5 1945 # XUSNation
5 1940 5 1940 # XNetherlandsNation
5 1940 5 1940 # XBelgiumNation
4 1941 4 1941 # XYugoslaviaNation
4 1940 6 1940 # XNorwayNation
10 1940 5 1941 # XGreeceNation
7 1940 11 1942 # XVichyNation
8 1944 5 1945 # XAlliedRomaniaNation
9 1943 5 1945 # XAlliedItalyNation
8 1940 5 1945 # XFreeFrenchNation
1 1939 8 1945 # XNationalistChineseNation
1 1930 12 1946 # XJapaneseNation
1 1939 8 1945 # XRedChineseNation
1 1939 8 1945 # XRedChicomNation
11 1939 5 1945 # XSovietNation
12 1941 8 1945 # XThaiNation
6 1941 6 1944 # XBulgarNation
6 1941 6 1944 # XCroatNation
9 1939 5 1945 # XIranNation
9 1939 5 1945 # XIraqNation
7 1936 4 1939 # XNatSpainNation
7 1936 4 1939 # XRepSpainNation
6 1941 6 1944 # XAlbanianNation
8 1942 5 1945 # XBrazilNation
10 1935 5 1936 # XEthiopiaNation
1 1939 8 1945 # XMongoliaNation
1 1939 8 1945 # XManchukoNation
10 1944 5 1945 # XAlliedBulgariaNation
1 1939 8 1945 # XWarlordsNation
9 1944 5 1945 # XAlliedFinlandNation
5 1945 5 1945 # XAlliedGermanyNation

---

Before 2.01, this timeline data was locked away, was hard-coded in the program EXE. It still is, but with 2.01, we have also exported this data to the moddable external file, timelines.dat. (If there is no timelines.dat, it will default to hard-coded values.)

In the forthcoming 2.01, for battle dates, you have:

[*]more informative error messages
[*]the means to extend the nation timelines

Of course, you might extend the nation timelines so far out that the game data -- the OOBs and ORGs -- don't support any absolutely ridiculously far out date. But then, with the -X NoEncryption option, you have the capability to mod your OOB & ORG (and other) files also however you please.

Have fun, modders!

[berto]

ORIGINAL: Big Ivan

Very nice work berto!

Thank you, it is appreciated.

And also the team, don't forget the team!

[Crossroads]

Nah, Berto does all the hard work, we just slack off and drink beer

[wings7]

berto and team, thank you, you guys are the best! [:D]

Patrick

[z1812]

Yes thanks for all your work. This is a great game system.

[Big Ivan]

ORIGINAL: berto

ORIGINAL: Big Ivan

Very nice work berto!

Thank you, it is appreciated.

And also the team, don't forget the team!

Opps, my bad. acknowledged the team in an edit of my post.

Sorry about that team.

Big Ivan

[Warhorse]

Here is a screenie to showcase a bit more of the 2D project, in town are SS, complete with runes, new heavy weapons, now unique icon for light, medium and heavy machine guns. Now also a infantry mortar icon for 60mm or less, and much, much more!! As mentioned, for the NATO/military symbology, instead of the original 90 or so, it is expanded to well over 3x that many now!! As for the graphical images, ie the little pictures, most units now have a unique image, no more sharing if it can be helped, ooh-rah 2D!!

[vonRocko]

Wow, great work! My thanks to everyone involved![&o]

[noissy]

A great undertaking, saying many thanks seems pathetic.
You guys have done an incredible job!

[Warhorse]

Thanks Peter!! Because we love it!!!

[Crossroads]

ORIGINAL: Crossroads

Here's hoping everyone's enjoying the new 2.01 UPDATE

It took a while coming, with lots of effort put into the front end applications this time around, Random Battle Generation and Dynamic Campaign Games for most part.

With that, there's a few changes to how things played out before. Going forward, as described in the Coder Diaries, and in preparation of the coming Modern Wars titles, the error handling and logging in general is being tightened. Common code base, remember. Happened here too.

For an example, with engagement Sizes, there's been a few questions around already.

In 2.00 and before, there was code in place to fish around for different levels if the game data doesn't support the requested level. There we identified problems where the code might go into an endless loop (effectively a freeze), or crash, or create mismatches: pitting a regiment vs. a brigade or even higher.

Now, there's logic in code now for a level playing ground. No more Regiments vs Bridgades. Or whatever the game engine picked from the selected countries' unit data.

On the other hand, the data in OOBs should now allow for level engagement sizes as well.

Therefore, one can't randomly generate any battle, any combatants, any size, with a perfect chance of success where there was success with earlier versions.

Some sets of choices will fail to work now. But others will do. Try try again.

There's a commitment for continuous improvement on unit data, this will take time.

For an example; the tricky thing with the Russians in that in 1943, there weren't a lot of division organizations. For armoured forces, they were mostly Corps, which you'll be given a notification (not an error) for in DCG's, but not in the Battle generator. So if someone picks regiment and tries to generate an armoured fight with regiments, it automatically will fail because Russian forces are organized into Brigades.

So, if there's a certain era, certain nations, consider creating specific oobs for specific eras ("Random Battle Packs"). Something that adds a bit more realism to random battles, and something the community could share among themselves.

Or, keep the 1.04 around if you so prefer. The good and the bad will work as they did before.

As for the development team effort. The coding for Middle East, then Vietnam will now proceed. For CS, any code additions will have wait until 2.1 is out, after them.

If there's a need, we can easily provide hot-fixes for data, though. And what you are seeing with 2.01, are data issues. In a couple of weeks time I will post a detail Developer Diary regarding the Battle Generator / DCG unit creation. Stay tuned.