A constructive discussion about stability, patches and bug fixes

[Dimitris]

Right, before diving into the core issue, let's get some fundamentals out of the way.

------------------------------------

* Every time this subject comes up, there is a suggestion about halting development of new features until a certain arbitrary theshold of "quality" is reached. This is not going to happen. Command has to keep moving forward and evolving in order to adapt to its evolving environment and market, both on the civ and pro sides. Putting a pause on new development would benefit exactly one group: Our competition. We're not about to hand them a freebie.

Bugs are a fact of software life. A big chunk of software development is about managing them while also moving forward (refresher). And yes, new features can (/will inevitably?) introduce new bugs. But without new features, software stops evolving to changing needs and requirements. (Why is almost no-one playing [computer] Harpoon right now?)

* Yes, Command is "unfinished". So is the OS of the device you're using at this moment. So is the web browser you are reading this thread with. (If you're not regularly updating your OS and your browser: Our condolences, and good luck). "Unfinished" is another term for continously being developed to improve with new features requested by the users and market, fixes to known issues as possible, and general improvements to the underlying codebase. There is an easy way to marvel at finished software: Browse your favorite abandonware site.

* We are very proud of the progress we've made on delivered quality the last 5-6 years or so. Anyone who's been with us for a while (such as Figeac himself, we hope) can recognize this. We have streamlined our internal development, collaboration and branching processes, expanded our internal QA team, *vastly* increased our testing coverage and automation (including internal scenarios for common issue cases), and introduced several new measures for preventing most common cases of failure. We've even consciously made "Snow Leopard"-like official releases (such as CMO v1.07) where the focus has explicitly been on improving the existing codebase and fixing issues rather than introducing new functionality. This is not to claim perfection by far; but to underline where we are: at a much better place than, say, 2020 or 21.

* Beta releases (both private and public) are inherently more "adventurous". New features are often present and still raw/unrefined, existing code is optimized or re-architected, new ideas are tried out etc. This almost invariably means new bugs, and sometimes old "friends" popping up again. This is part of the reason they are not for everyone. We would like to think our official releases have been pretty stable, particularly since v1.07, but this is a judgement call that we prefer that the player community make for themselves.

-------------------------------------

Now then, onto the main point: Why are there voiced concerns about long-persistent and regressing issues? Particularly since internally we are, in fact, checking for such problems before each release?

My suspicion is that the root cause of the problem is this: That the scenarios used internally to check for problems, and the scenarios "in the wild" where the long-persistent and regressing issues mostly manifest themselves, are not one and the same.

If we accept this as a major part of the problem, then a major part of the solution presents itself: Make these scenarios one and the same.

For this reason, I am inviting those who have experienced such issues (Figeac as the OP but also anyone else) to share their specific scenarios/saves with us. We will take them and try to add them to our existing internal test suite. (I say "try" because not every simulated predicament lends itself to instrumentation for efficient testing). If we are able to, these will become a permanent feature of our pre-release testing pipeline.

Does this guarantee that the problem in question will be eradicated and/or not resurface? No, but it does mean that the dev & QA teams will at least have an opportunity to catch it prior to it re-appearing in the wild and causing the heartache so obvious in this thread.

Open to reasonable suggestions.

[Nikel]

This time I was the "suggester", I think that previously it was kobu, after tiny patch release. Sorry, if it was not the case.


Suggestion done on some assumptions:

- That there will be a Command v3 in the future. Halting the development only for CMO, not for the next release.

- Enough devs to keep 2 versions for some time. Lets imagine you are 10. 1 or 2 , the best "bug fixers", will work on CMO just doing this job, fixing a moving target is more difficult than trying to fix a static one. The rest will continue with the development of Command v3.

- Future features and additions are not infinite. You have to reserve cool new features for the next release.


Perhaps this is just nonsense, because you are not enough people or because there are not "bug fixers" or you have so many ideas that there is really no problem in adding more stuff to CMO

[Varangian]

Hi All,
I am a bit reluctant to put my two-cents into this discussion but I would like to provide a viewpoint from a more casual user and ask for guidance.

I am a long-time admirer of CMO and I purchased all the DLC, usually the day of release. I recognize the issues with such complex software, but I am not a software or project engineer. I have some experience coding for particle physics experiments.

I said I am an "admirer" because I do not know how to label my experiences "playing" CMO. I don't have the time to play frequently, but when I do play it usually ends with awe at the details in the game, but also frustration and confusion. Two examples,

- when Fail Safe was first released I played the first scenario ("Down bird"?) and was having a blast, when a pop-up notified me of "Fuse Exceptions" and I was instructed how to send a bug report (which I did). Afterwards, I did not know how to proceed. Should I continue or was my scenario now screwed up? Should I try again or a different scenario because this one was bugged. Should I wait for a fix before playing CMO again? I decided (rightly or wrongly) to leave off playing for some time and I hoped the bugs were being fixed.

- I recently updated to the latest version and played Northern Inferno, "goblins on the doorstep" (opposed transit of UK boomer to patrol). I continually got sunk, but it was great fun. However, I recently read that sonobuoys may not be working (which are kind of important in that scenario). I am left wondering if my failures in the game were caused by my inexperience or bugs in the game? I would like to try again, but don't feel it would be worthwhile until sonobuoys are fixed. Strangely, I would not have known that there was a problem if I did not check the tech support discussions.

I am not describing my experiences to vent or complain, but to seek guidance on how to realistically move forward with playing and enjoying CMO in the light of my experiences.

What do players need to do to have a good experience if we just want to sit down and play when the game is at some random point in the development cycle? What should be our expectations?

Thanks for your amazing work on this game,
Barry

[thewood1]

The fuse and exception reports lately have been creating false perceptions. Generally, the game runs fine after an exception. In fact, its only relatively recently the devs put the fuse limits in. Before that, no one even knew an error occurred. Now there are people scouring the logs for ANY error and have a "sky is falling" message about the game on every logged error.

My experience after playing over 15 years...just play the game. Yes, now and then, a critical error might pop up. I play 10-20 hours a week. I can count on one hand then number of times I have had a crash over the last two years. I can count on two hands times I got an error that completely screwed up a scenario over that same period. And I play a lot. CMNAO/CMO has had hundreds of beta builds. I have only had to roll back to stable twice in the ten or so years.

The best experience you can have is play the game. Stop reading the tech forums. If something breaks, report it and move on. There are people reporting things now that don't understand the game or the development cycle of complex software. Public betas are a good and bad thing. Legitimate critical issues get all mixed up with obsessive reporting and give the impression that game is completely broken.

I want to emphasize that bugs do happen. And critical game play bugs do happen. The only way to stop them is completely stop the FREE growth of the game. If you can't separate the real bugs from the every day issues reported in an agile development process, stop playing betas and stop reading the forums. Its the only way I know of. Again, as I know people will misconstrue my words, bugs happen and should be reported. But the scouring of the logs and massive over-reporting on every little issue leaves the impression the devs are just shoveling stuff out the door. They aren't.

The nuclear option is get to a point you like with the game and stop updating. Thats what some people are asking the devs to do without realizing they can already do that themselves.

[blu3s]

Exceptions impact performance, which is why we have fuse exceptions. So I’m fine with a simple report about something happening in a specific scenario. Also, this could sometimes cause unwanted behavior.

We’re deviating from the open question Dimitris raised. Which specific scenarios or subjects would you like to see covered in the pipeline test?

[Dimitris]

- I recently updated to the latest version and played Northern Inferno, "goblins on the doorstep" (opposed transit of UK boomer to patrol). I continually got sunk, but it was great fun. However, I recently read that sonobuoys may not be working (which are kind of important in that scenario). I am left wondering if my failures in the game were caused by my inexperience or bugs in the game? I would like to try again, but don't feel it would be worthwhile until sonobuoys are fixed. Strangely, I would not have known that there was a problem if I did not check the tech support discussions.

That specific issue appeared in one of the recent public betas, and was fixed on the very next one. We've been very clear that public betas are not for everyone, and things like that can happen. For anyone who prefers to avoid the anguish of such cases, sticking strictly with the official releases is highly recommended.