A constructive discussion about stability, patches and bug fixes

[Dimitris]

Right, before diving into the core issue, let's get some fundamentals out of the way.

------------------------------------

* Every time this subject comes up, there is a suggestion about halting development of new features until a certain arbitrary theshold of "quality" is reached. This is not going to happen. Command has to keep moving forward and evolving in order to adapt to its evolving environment and market, both on the civ and pro sides. Putting a pause on new development would benefit exactly one group: Our competition. We're not about to hand them a freebie.

Bugs are a fact of software life. A big chunk of software development is about managing them while also moving forward (refresher). And yes, new features can (/will inevitably?) introduce new bugs. But without new features, software stops evolving to changing needs and requirements. (Why is almost no-one playing [computer] Harpoon right now?)

* Yes, Command is "unfinished". So is the OS of the device you're using at this moment. So is the web browser you are reading this thread with. (If you're not regularly updating your OS and your browser: Our condolences, and good luck). "Unfinished" is another term for continously being developed to improve with new features requested by the users and market, fixes to known issues as possible, and general improvements to the underlying codebase. There is an easy way to marvel at finished software: Browse your favorite abandonware site.

* We are very proud of the progress we've made on delivered quality the last 5-6 years or so. Anyone who's been with us for a while (such as Figeac himself, we hope) can recognize this. We have streamlined our internal development, collaboration and branching processes, expanded our internal QA team, *vastly* increased our testing coverage and automation (including internal scenarios for common issue cases), and introduced several new measures for preventing most common cases of failure. We've even consciously made "Snow Leopard"-like official releases (such as CMO v1.07) where the focus has explicitly been on improving the existing codebase and fixing issues rather than introducing new functionality. This is not to claim perfection by far; but to underline where we are: at a much better place than, say, 2020 or 21.

* Beta releases (both private and public) are inherently more "adventurous". New features are often present and still raw/unrefined, existing code is optimized or re-architected, new ideas are tried out etc. This almost invariably means new bugs, and sometimes old "friends" popping up again. This is part of the reason they are not for everyone. We would like to think our official releases have been pretty stable, particularly since v1.07, but this is a judgement call that we prefer that the player community make for themselves.

-------------------------------------

Now then, onto the main point: Why are there voiced concerns about long-persistent and regressing issues? Particularly since internally we are, in fact, checking for such problems before each release?

My suspicion is that the root cause of the problem is this: That the scenarios used internally to check for problems, and the scenarios "in the wild" where the long-persistent and regressing issues mostly manifest themselves, are not one and the same.

If we accept this as a major part of the problem, then a major part of the solution presents itself: Make these scenarios one and the same.

For this reason, I am inviting those who have experienced such issues (Figeac as the OP but also anyone else) to share their specific scenarios/saves with us. We will take them and try to add them to our existing internal test suite. (I say "try" because not every simulated predicament lends itself to instrumentation for efficient testing). If we are able to, these will become a permanent feature of our pre-release testing pipeline.

Does this guarantee that the problem in question will be eradicated and/or not resurface? No, but it does mean that the dev & QA teams will at least have an opportunity to catch it prior to it re-appearing in the wild and causing the heartache so obvious in this thread.

Open to reasonable suggestions.

[Nikel]

This time I was the "suggester", I think that previously it was kobu, after tiny patch release. Sorry, if it was not the case.


Suggestion done on some assumptions:

- That there will be a Command v3 in the future. Halting the development only for CMO, not for the next release.

- Enough devs to keep 2 versions for some time. Lets imagine you are 10. 1 or 2 , the best "bug fixers", will work on CMO just doing this job, fixing a moving target is more difficult than trying to fix a static one. The rest will continue with the development of Command v3.

- Future features and additions are not infinite. You have to reserve cool new features for the next release.


Perhaps this is just nonsense, because you are not enough people or because there are not "bug fixers" or you have so many ideas that there is really no problem in adding more stuff to CMO

[Varangian]

Hi All,
I am a bit reluctant to put my two-cents into this discussion but I would like to provide a viewpoint from a more casual user and ask for guidance.

I am a long-time admirer of CMO and I purchased all the DLC, usually the day of release. I recognize the issues with such complex software, but I am not a software or project engineer. I have some experience coding for particle physics experiments.

I said I am an "admirer" because I do not know how to label my experiences "playing" CMO. I don't have the time to play frequently, but when I do play it usually ends with awe at the details in the game, but also frustration and confusion. Two examples,

- when Fail Safe was first released I played the first scenario ("Down bird"?) and was having a blast, when a pop-up notified me of "Fuse Exceptions" and I was instructed how to send a bug report (which I did). Afterwards, I did not know how to proceed. Should I continue or was my scenario now screwed up? Should I try again or a different scenario because this one was bugged. Should I wait for a fix before playing CMO again? I decided (rightly or wrongly) to leave off playing for some time and I hoped the bugs were being fixed.

- I recently updated to the latest version and played Northern Inferno, "goblins on the doorstep" (opposed transit of UK boomer to patrol). I continually got sunk, but it was great fun. However, I recently read that sonobuoys may not be working (which are kind of important in that scenario). I am left wondering if my failures in the game were caused by my inexperience or bugs in the game? I would like to try again, but don't feel it would be worthwhile until sonobuoys are fixed. Strangely, I would not have known that there was a problem if I did not check the tech support discussions.

I am not describing my experiences to vent or complain, but to seek guidance on how to realistically move forward with playing and enjoying CMO in the light of my experiences.

What do players need to do to have a good experience if we just want to sit down and play when the game is at some random point in the development cycle? What should be our expectations?

Thanks for your amazing work on this game,
Barry

[thewood1]

The fuse and exception reports lately have been creating false perceptions. Generally, the game runs fine after an exception. In fact, its only relatively recently the devs put the fuse limits in. Before that, no one even knew an error occurred. Now there are people scouring the logs for ANY error and have a "sky is falling" message about the game on every logged error.

My experience after playing over 15 years...just play the game. Yes, now and then, a critical error might pop up. I play 10-20 hours a week. I can count on one hand then number of times I have had a crash over the last two years. I can count on two hands times I got an error that completely screwed up a scenario over that same period. And I play a lot. CMNAO/CMO has had hundreds of beta builds. I have only had to roll back to stable twice in the ten or so years.

The best experience you can have is play the game. Stop reading the tech forums. If something breaks, report it and move on. There are people reporting things now that don't understand the game or the development cycle of complex software. Public betas are a good and bad thing. Legitimate critical issues get all mixed up with obsessive reporting and give the impression that game is completely broken.

I want to emphasize that bugs do happen. And critical game play bugs do happen. The only way to stop them is completely stop the FREE growth of the game. If you can't separate the real bugs from the every day issues reported in an agile development process, stop playing betas and stop reading the forums. Its the only way I know of. Again, as I know people will misconstrue my words, bugs happen and should be reported. But the scouring of the logs and massive over-reporting on every little issue leaves the impression the devs are just shoveling stuff out the door. They aren't.

The nuclear option is get to a point you like with the game and stop updating. Thats what some people are asking the devs to do without realizing they can already do that themselves.

[blu3s]

Exceptions impact performance, which is why we have fuse exceptions. So I’m fine with a simple report about something happening in a specific scenario. Also, this could sometimes cause unwanted behavior.

We’re deviating from the open question Dimitris raised. Which specific scenarios or subjects would you like to see covered in the pipeline test?

[Dimitris]

- I recently updated to the latest version and played Northern Inferno, "goblins on the doorstep" (opposed transit of UK boomer to patrol). I continually got sunk, but it was great fun. However, I recently read that sonobuoys may not be working (which are kind of important in that scenario). I am left wondering if my failures in the game were caused by my inexperience or bugs in the game? I would like to try again, but don't feel it would be worthwhile until sonobuoys are fixed. Strangely, I would not have known that there was a problem if I did not check the tech support discussions.

That specific issue appeared in one of the recent public betas, and was fixed on the very next one. We've been very clear that public betas are not for everyone, and things like that can happen. For anyone who prefers to avoid the anguish of such cases, sticking strictly with the official releases is highly recommended.

[Nikel]

- when Fail Safe was first released I played the first scenario ("Down bird"?) and was having a blast, when a pop-up notified me of "Fuse Exceptions" and I was instructed how to send a bug report (which I did). Afterwards, I did not know how to proceed. Should I continue or was my scenario now screwed up? Should I try again or a different scenario because this one was bugged. Should I wait for a fix before playing CMO again? I decided (rightly or wrongly) to leave off playing for some time and I hoped the bugs were being fixed.

- I recently updated to the latest version and played Northern Inferno, "goblins on the doorstep" (opposed transit of UK boomer to patrol). I continually got sunk, but it was great fun. However, I recently read that sonobuoys may not be working (which are kind of important in that scenario). I am left wondering if my failures in the game were caused by my inexperience or bugs in the game? I would like to try again, but don't feel it would be worthwhile until sonobuoys are fixed. Strangely, I would not have known that there was a problem if I did not check the tech support discussions.

I am not describing my experiences to vent or complain, but to seek guidance on how to realistically move forward with playing and enjoying CMO in the light of my experiences.

This is my advice as a fellow player regarding the 2 examples you posted, and guessing that you still want to try the last betas.

- Untick the "Log debug information to file" box in the options. The exceptions on the screen will not happen.

D.gif (134.33 KiB) Viewed 971 times

- Perhaps you will still notice that a function is broken in the last beta, but want to play a scenario. Then you may use the previous build or beta. You can do this easily if you have not overwritten the previous exes, something that I do not recommend. To avoid overwriting, before applying or extracting the last beta, you have to rename "Command.exe" to "Command 1877.exe". This is just an example with the last builds to get the idea. In the same folder you may have several builds of command renamed this way. When you want to use another one, you have to rename again because only "Command.exe" will work. Command.exe then is renamed to "Command 1892.exe" and the one you want to use "Command.exe"


Hope this helps.

[Dimitris]

Repeating my open invitation to Figeac (as OP) but also to anyone else that shares his experience and frustrations:

Give us your persistently problematic scenarios (new thread per each on Tech Support, so that we can track them). Show us in detail how to reproduce the issues. HELP US HELP YOU.

[scott1393]

And nobody is talking about the new naval formation editor...

Being new it must have glitches, but the look is modern and fantastic. A bit dark perhaps, cannot see clearly the cells.

Congrats to the devs team



NF.gif

im on build 1892 and when i F4 for the editor im still getting the old formation editor. where did this one come from i went back through the last 2 beta updates and i can find any mention of the new editor.

scott

[Nikel]

im on build 1892 and when i F4 for the editor im still getting the old formation editor. where did this one come from i went back through the last 2 beta updates and i can find any mention of the new editor.

scott

It is explained in the last build thread.

ADDED: New dedicated naval formation editor: Select any naval group and press Shift+F4: https://i.ibb.co/fYxVctRK/image.png
The new naval formation allows easier placement of escorts around a HVU, as well as new standard formation templates: https://i.ibb.co/FLr3xp2y/image.png

[scott1393]

im on build 1892 and when i F4 for the editor im still getting the old formation editor. where did this one come from i went back through the last 2 beta updates and i can find any mention of the new editor.

scott

It is explained in the last build thread.

ADDED: New dedicated naval formation editor: Select any naval group and press Shift+F4: https://i.ibb.co/fYxVctRK/image.png
The new naval formation allows easier placement of escorts around a HVU, as well as new standard formation templates: https://i.ibb.co/FLr3xp2y/image.png

thanks wouldnt you know its the first line of the thread. thanks again.

scott

[DaveFromCTX]

My suspicion is that the root cause of the problem is this: That the scenarios used internally to check for problems, and the scenarios "in the wild" where the long-persistent and regressing issues mostly manifest themselves, are not one and the same.

I'm glad this was brought up. Not sure how many other users can say this but I've been playing the exact same scenario that I've rebuilt a handful of times for the past 5 years. It is the primary reason I do not make bug reports. That and the scenario I use has been added on to so many times that it's complexity is like going to cause more confusion than it eliminates.

[Dimitris]

My suspicion is that the root cause of the problem is this: That the scenarios used internally to check for problems, and the scenarios "in the wild" where the long-persistent and regressing issues mostly manifest themselves, are not one and the same.

I'm glad this was brought up. Not sure how many other users can say this but I've been playing the exact same scenario that I've rebuilt a handful of times for the past 5 years. It is the primary reason I do not make bug reports. That and the scenario I use has been added on to so many times that it's complexity is like going to cause more confusion than it eliminates.

So, if there are any persistent or recurring issues on it, can we have a look ?

[Dimitris]

Anyone ?

[Nikel]

I would add a benchmark with the benchmark scenario, testing that:

- There are no exceptions in the log.

- There is no significant slowdown, and if there is, is explainable by the additions or changes you wanted to do.


Also a gameplay by a human, not an automatic test. That assures the basic CMO functions work. This would have identified the recent sonobuoys problem for example.

This gameplay should also provide a clean exception log.

[Dimitris]

I would really like to hear from the OP (Figeac).

[tiag]

Perhaps extend unit tests + integration tests and possibly flags for changes to disable new code if found buggy later when released?
(possible flags: Performance, Compatible Changes, Priority Fixes, and Non-Compatible Changes <- this is how we did in Falcon 4 BMS at least).

[Knightpawn]

this is how we did in Falcon 4 BMS at least.

PAYING MY RESPECTS