When?

[Shannon V. OKeets]

ORIGINAL: Evildan

So what happens after Steve is forced to retire and the CWIF game hasn't been published yet?

"If this man should fall...
who will pick up the flag in his stead?!"

To quote Mark Twain: "The rumors of my death have been greatly exaggerated."

[Skanvak]

I think the reasonnable date is to give no date due to the actual developpement model. I share the feeling that giving no expectation is better than failed expectations.

I am happy that the rumour about Steve's death were exagerated.

So let developpement be as we all want a finished game.

[Sewerlobster]

ORIGINAL: macgregor

ORIGINAL: Blorsh
At 1998 Wif annual was announced the CWIF soon, so be patient,the CWIF will became MWIF someday, I hope so.

Yes I know. My dog; CWIF is now 12 years old. I get anxious, yes, but it's for all his sake.

Macgregors gaming group: Won't eat until Mwif is released.

[macgregor]

Well you're partially right. At least Steve gives us enough to maintain body weight. When I heard that TOAW's long-awaited patch was going to deny larger scenarios even minimally functional naval unit attributes(when it ever shows up -MWiF may get here first) I switched avatars. To me all other work on that game is tantamount to rearranging deck chairs on the Titanic.

[Shannon V. OKeets]

March 1, 2010 Status Report for Matrix Games’ MWIF Forum


Accomplishments of February 2010

Project Management
I monitored all the threads in the MWIF World in Flames forum daily.

Hardware and Software
I spent virtually all of February replacing assembler routines with equivalent Pascal routines. This is still a work in progress but it has improved the performance of the Delphi 2010 IDE (Interactive Development Environment) when executing MWIF in debug mode. Happily, I am now able to examine internal values of variables while the program is running. Things aren’t 100% what they had been when I was running Delphi 2007, but they are ~80% now. Once I get the rest of assembler code replaced with pure Pascal, I am hopeful of getting my debugging efficiency back to where it was a couple of months ago.

Why am I removing the assembler code? Well here is a short explanation from Zorachus99:

No wonder you are having trouble with your IDE.
1) Some registers are being quietly deprecated, and a lot of the old ones don't work the same in a 32 vs 64 bit OS.
2) New registers are being added to processors for a variety of reasons
3) Some registers are being re-purposed to do different things.

The short and long as a result:

1) OS (Microsoft) vendors don't want you playing with the CPU directly and go to great lengths to prevent it with emulators (which fail regularly). This is primarily because 16 bit != 32 bit != 64 bit != 128 bit registers.
2) Theoretically you could live off emulators, but it's better to move up a level and abstract routines, not directly handling hardware, rather than try to drive the hardware at maximum speed.


Registers are the storage locations used by the CPU. Indeed, they are a physical part of the CPU. I wrote a couple of games in assembler for the 8086 CPU, which evolved into a family of CPUs known collectively and the X86 CPUs. If you are running Win7, you’ll see that your file directory has two folders for program files: Program Files and Program Files (x86). The latter is for programs written for the older x86 family of CPUs.

I have some direct evidence that the problems I have had with Delphi 2010 relate to the CWIF code that is written in assembler. In technical terms, assembler works with the CPU registers directly, and the CWIF code modified the call stack directly. As Zorachus99 said, the guys who write the operating system code do not want application programmers messing around with those. From a practical point of view, if they don’t want you (the application programmer) to do that, then you shouldn’t, since it is likely that you will not be informed about changes they make in the operating system. If they change something in the operating system that the application program’s assembler code was relying on, well, then the program crashes.

Pascal is a strongly typed language - it has a lot of rules about what can and cannot be done. While you can add ‘St’ to ‘eve’ and get ‘Steve’, or add 3 to 4 and get 7, you cannot add 6 to ‘air units’ and expect to get 6 air units. The compiler checks to see that everything makes sense. If any of the rules it imposes are violated, the compiler displays an error message when the program is compiled, or at least a warning message.

Assembler works with binary strings (e.g., 0111001) and uses them for both variables and procedures. In assembler, variables are untyped - ‘this’. A procedure - “do that” (e.g., sort by unit type, find all bombers) is also a binary string. In assembler you can add 6 to air units and the compiler will not complain. Of course the program will crash in strange ways when it executes that instruction, but there will be no warnings from the compiler about the faulty code.

For those of you who do not understand any of this, let me try a simple analogy. Pascal has rules, like the rules of the road for cars. It stops the programmer from violating the rules of the road, thereby preventing numerous crashes. Assembler, however, has no rules. The cars can drive wherever they like, however they like. With assembler, you can go the wrong direction on one-way streets or exceed the speed limit on S curves. That is fine if the driver/programmer is very careful and it helps if you are the only car on the road.

When we introduce different CPU chips (e.g., dual core and quad core), there can be more than “1 car on the road”. Threading has a similar effect. The operating systems that you are use to running keep several program executing simultaneously and they do that using complex software that works down at the level of the CPU registers. An application written in assembler relies on the CPU registers and call stack being available according to certain rules. If the operating system vender (i.e, Microsoft) or the compiler vendor (Embarcadero) changes those rules, then the program has to be modified to obey the changed behavior.

So, MWIF contained a fair amount of assembler code. The new CPUs and operating systems don’t process that code as the programmer intended/expected. More threatening, Embarcadero has announced that integrating assembler code into Pascal code with be deprecated in future releases of Delphi.

While I was able to replace a lot of the assembler code easily, there were a few places where a one-to-one replacement could not be done. The assembler code more or less took ‘this’ and did ‘that’ to it. Pascal wants all those variables and procedures typed. While the assembler code worked with all variable types and all procedure types, in Pascal separate code fragments are necessary for each variation in the program. There were ~1730 references to one assembler procedure in MWIF. Each of those single line references requires 8-10 lines of new code. Over the course of February I have gotten that count down to 590, while also cleaning up all the other miscellaneous bit of assembler code. I should be able to purge MWIF of the last traces of assembler in the next week. Note that once all the assembler is gone, I will have added ~14,000 lines of new code to the program.

Beta Testing
Right now the beta testers are running into fatal crashes caused by Theme Engine 9.10. But I am assuming that software library is innocent until after I get all the assembler code removed. If it still causes trouble once the assembler code is gone, I’ll look into rewriting some (as little as possible) of the Theme Engine source code - i.e., I’ll debug it for MWIF.

I released versions 4.00.06 and 4.00.07 of MWIF to the beta testers in February. The primary goal of these versions was to let the beta testers see if I screwed up anything that use to work before my hundreds of edits replacing assembler calls.

Saved Games
Nothing new.

Map and Units
Nothing new other than some more naval unit writeups from Rob/Warspite.

Scenarios and Optional Rules
Nothing new.

MWIF Game Engine and CWIF Conversion
Assembler code replaced with Pascal code.

Player Interface
Nothing new.

Internet - NetPlay
Nothing new.

PBEM
Nothing new.

Artificial Intelligence (AI)
Nothing new.

Player’s Manual
Nothing new.

Tutorials, Training Videos, and Context Sensitive Help
Nothing new.

Historical Video, Music, and Sound Effects
Nothing new.

Marketing
Nothing new.

Communications
Nothing new.

[CSSS]

Sounds like a Bome Marrow transplant?

[norvandave]

Steve, in a weird way I guess it was a good thing that you upgraded to Delphi 10 and found these problems now. It probably would have been harder to fix down the road. Perhaps some users would have experienced problems due to the assembler code.

It seems that the path is clear now to finish the game sometime this year. <fingers crossed>

Keep up the fight. Waiting patiently. I know it will be worth the wait. I'm sure as much as we want the game, you want to finish it much more.

-DAVE

[vonpaul]

Thanks for the update Steve.

Norvandave, I think we should have realistic expectations about the release date.

[oscar72se]

ORIGINAL: Shannon V. OKeets
When we introduce different CPU chips (e.g., dual core and quad core), there can be more than “1 car on the road”. Threading has a similar effect. The operating systems that you are use to running keep several program executing simultaneously and they do that using complex software that works down at the level of the CPU registers. An application written in assembler relies on the CPU registers and call stack being available according to certain rules. If the operating system vender (i.e, Microsoft) or the compiler vendor (Embarcadero) changes those rules, then the program has to be modified to obey the changed behavior.

I don't have any experience of Delphi 2010, even though I had a brief encounter with Delpi in the late 90s [;)] IMHO whilst threading is a powerful tool to make asynchrounous calls "off the main thread" it also is a maaaajor source of what I like to call "occult bugs" [:)] The problem is that most debuggers have a hard time attaching themselves to external processes, which means that errors "go unhandled" from the main threads POV. Bugs created this way and that don't cause immediate crashes have a tendency of being super hard to find. I have found that a very good approach is to write separate applications that are only used to test the different routines that are to be executed in its' own thread.

Once or twice in my life I have stumbled upon tasks similar to your "migrate-50-year-old-code" to routines that actually makes any sense and that fits into a modern object oriented world. Each time it resulted in me rewriting everything from scratch. I think I have a vague understanding of the scope of your task, so your feat doesn't go entirely unnoticed...

Keep fighting Steve!
[:)]

Regards,
Oscar

[yvesp]

ORIGINAL: Shannon V. OKeets
While I was able to replace a lot of the assembler code easily, there were a few places where a one-to-one replacement could not be done. The assembler code more or less took ‘this’ and did ‘that’ to it. Pascal wants all those variables and procedures typed. While the assembler code worked with all variable types and all procedure types, in Pascal separate code fragments are necessary for each variation in the program. There were ~1730 references to one assembler procedure in MWIF. Each of those single line references requires 8-10 lines of new code. Over the course of February I have gotten that count down to 590, while also cleaning up all the other miscellaneous bit of assembler code. I should be able to purge MWIF of the last traces of assembler in the next week. Note that once all the assembler is gone, I will have added ~14,000 lines of new code to the program.

Looks like these assembler procedures were used to do the same thing to different types of data ?
Don't you have genericity in Delphi ?

Yves

[Josh]

" I spent virtually all of February replacing assembler routines with equivalent Pascal routines. This is still a work in progress but it has improved the performance of the Delphi 2010 IDE (Interactive Development Environment) when executing MWIF in debug mode. Happily, I am now able to examine internal values of variables while the program is running. Things aren’t 100% what they had been when I was running Delphi 2007, but they are ~80% now. Once I get the rest of assembler code replaced with pure Pascal, I am hopeful of getting my debugging efficiency back to where it was a couple of months ago." [&o]

Gosh, sounds like one step forward.. and two back [:D] But seriously, that's a *lot* of work.
Thx for the update Steve.

[composer99]

It looks like it's just as well that this problem happened to Steve (as frustrating as it may have been to correct). Otherwise it might have passed on unnoticed and happened to us.

[Zorachus99]

Glad to hear that debug isn't going crazy any longer. I'm sure there was intense internal pressures on your PC to cough up info for the debugger that most likely either wasn't there or couldn't be checked.

At least you are moving toward hardware, and potentially OS independance with your project.

Good luck, and VOG free sunny days coming to you soon - minus the occasional tidal wave.

Cheers!

[CarnivalBizarre]

Uh...

[Shannon V. OKeets]

ORIGINAL: oscar72se

ORIGINAL: Shannon V. OKeets
When we introduce different CPU chips (e.g., dual core and quad core), there can be more than “1 car on the road”. Threading has a similar effect. The operating systems that you are use to running keep several program executing simultaneously and they do that using complex software that works down at the level of the CPU registers. An application written in assembler relies on the CPU registers and call stack being available according to certain rules. If the operating system vender (i.e, Microsoft) or the compiler vendor (Embarcadero) changes those rules, then the program has to be modified to obey the changed behavior.

I don't have any experience of Delphi 2010, even though I had a brief encounter with Delpi in the late 90s [;)] IMHO whilst threading is a powerful tool to make asynchrounous calls "off the main thread" it also is a maaaajor source of what I like to call "occult bugs" [:)] The problem is that most debuggers have a hard time attaching themselves to external processes, which means that errors "go unhandled" from the main threads POV. Bugs created this way and that don't cause immediate crashes have a tendency of being super hard to find. I have found that a very good approach is to write separate applications that are only used to test the different routines that are to be executed in its' own thread.

Once or twice in my life I have stumbled upon tasks similar to your "migrate-50-year-old-code" to routines that actually makes any sense and that fits into a modern object oriented world. Each time it resulted in me rewriting everything from scratch. I think I have a vague understanding of the scope of your task, so your feat doesn't go entirely unnoticed...

Keep fighting Steve!
[:)]

Regards,
Oscar

About threading, ...

I am not a fervent fan of threading, but it does have its uses. I see it being useful/essential for monitoring the internet during NetPlay and, to a lesser extent, during PBEM. In both those cases the code executing in a thread separate from the main program's thread will be less than 100 lines.

The more serious use of a second thread will be during AIO play, where the AI Opponent will be figuring out its move while the human player fumbles around with the keyboard, mouse, and his wetware. Your concern about the debugger having trouble tracking down bugs in the second thread is something I'll pay attention to there - now that you have warned about it. Thanks.

[Shannon V. OKeets]

ORIGINAL: yvesp

ORIGINAL: Shannon V. OKeets
While I was able to replace a lot of the assembler code easily, there were a few places where a one-to-one replacement could not be done. The assembler code more or less took ‘this’ and did ‘that’ to it. Pascal wants all those variables and procedures typed. While the assembler code worked with all variable types and all procedure types, in Pascal separate code fragments are necessary for each variation in the program. There were ~1730 references to one assembler procedure in MWIF. Each of those single line references requires 8-10 lines of new code. Over the course of February I have gotten that count down to 590, while also cleaning up all the other miscellaneous bit of assembler code. I should be able to purge MWIF of the last traces of assembler in the next week. Note that once all the assembler is gone, I will have added ~14,000 lines of new code to the program.

Looks like these assembler procedures were used to do the same thing to different types of data ?
Don't you have genericity in Delphi ?

Yves

Delphi permits some generality for argument passing, but Pascal is stronglytyped. So, when a function is passed, the compiler wants to have information about the argument list so it can check whether the passed function matches. The result is that you need more than one 'universal' function. Here is some of the new code in MWIF:

// ****************************************************************************
// The next 6 routines pass Func or Proc as parameters. In order for the Func
// and Proc to execute correctly, they cannot use any variables that are local.
// All referenced variables have to either be a parameter in the argument list
// of the called Func/Proc or else be global. The module PassedParameters
// defines global variables which are used expressly for this purpose. Care
// must be taken to not use these in recursive calls or nested calls that use
// the same variable.
// ****************************************************************************
function DoesAnyStack(const Func: TStackFunc): Boolean;
function WhichStackDoes(const Func: TStackFunc): TMapStack;
function DoesAnyUnit(const Func: TUnitFunc): Boolean;
procedure ProcessEachUnit(const Proc: TUnitProc);
procedure ProcessEachUnitReverse(const Proc: TUnitProc);
function CountEachUnitReverse(const Func: TUnitFunc): Integer;

...

TUnitProc = procedure(var U);
TUnitFunc = function(var U): Boolean;
TStackFunc = function(var S): Boolean;
THexFunc = function(const Hex): Boolean;
TCoastalProc = procedure(var CD);

[SamuraiProgrmmr]

If I might be so bold as to add, Delphi Programmers like strict typing. It prevents all kinds of problems from ever being a problem. It does make you have to work a little harder on some things, but you don't have to worry at all about an entire class of bugs.

[macgregor]

Thanks for the update Steve. If I'm reading this correctly ...there was a bug that required a reformatting of a lot of the game code, eliminating this 'assembler code' for this new generation of Delphi.

I'm okay with being back to where we were 2 months ago if it means we can now move forward. I imagine there are parts of this game that were written a long time ago(in computer years). Am I to believe the game has been sufficiently re-written to the current standard to move forward full steam now?

What's seemingly not being mentioned is that the game has been suffering from some kind of a major bug that is halting all progress up to now. Right? I mean, the workload hasn't changed. Am I waiting for a 'eureka' moment? Or did one just occur? Someone besides Steve reply if possible. He's got enough to do.

[paulderynck]

There is no major bug. There was a major overhaul of the development platform that caused a number of unanticipated side effects that took time away from development over the last two months. There is further development and bug fixing still to be done. Details regarding the further development outstanding are provided in previous monthly updates.

[Joseignacio]

ORIGINAL: paulderynck

There is no major bug. There was a major overhaul of the development platform that caused a number of unanticipated side effects that took time away from development over the last two months. There is further development and bug fixing still to be done. Details regarding the further development outstanding are provided in previous monthly updates.

I understand the same.

Keep up the work, Steve!