Do you think game developers actually play their games?

[aaatoysandmore]

I often wonder sometimes if game developers actually play thier games. Since we find so many bugs and flaws after release. I mean some things are so apparent that they couldn't have been missed by the developer if he actually played the game. Like in the Total Wars nearly all of them at the beginning have the general charging to his death at the beginning of the battle. How could a developer have missed that?

Can you name some other things developers shouldn't have missed IF they actually played their own games?


I also think games should have AI ratings on the description. From easy to impossible. It's something I'm always in wonderment about before I buy a game. How good is the AI. It needs some kind of grade from the BETA testers not the developers. What do you think?

[Gilmer]

I don't think the ones actually writing the code have time to play their own games during development. That's why they get beta-testers, I believe, while they are constantly working on stuff they already know is broken. Maybe after the game is out, they get a chance.

[NikiAlex]

I think that developers are too busy to complete tasks on time rather to play the games.
Producers and designers should be forced to play all kind of games. This might help the creative part.
Regarding the quality I don't think it’s a developers fault.
IMO poor QAs and the overall project timeline and planning are primary contributors to game breaking bugs.

[Mobeer]

I think there are two things:

- do the games get adequately tested, and then if issues found, are they fixed?

I get the impression (as an outsider) that there is too much use of "beta-testing", rather than real testing. By which I mean testing in a structured organised way. As an example, think of a (fixed) 200-turn campaign game that takes 30 minutes per turn if played by a human. I'd like to know what proportion of the "beta testers" will:
- autorun through 180 turns to try the end game
- play zero turns, but just try out most of the UI
- ignore the campaign, just try making and running scenarios
vs. the number of "beta testers" who will just start a campaign with turn 1 and play the first 40 turns in the test period.

There was a published list of issues for Wing Commander III that was interesting. QA found big issues, they were not fixed as too expensive\time consuming to fix, then customers complained, as predicted by the QA team. Just finding issues is only half a job; they have to be assessed, prioritised and if necessary resolved.


- do the developers see \ care about the way the games are actually played?

There have certainly been games where the issues were not so much with bugs but rather with design choices that the developers failed to see the full implications of.
I remember Age of Kingdoms, where the most potent unit was the Frankish town hall - a defensive, civilian structure that out-gunned all enemies early on. Players built a series of these buildings as a way of rushing an opponent. Developers made this structure powerful without ever predicting it would be used offensively.

Or what about the Command and Conquer tactic of surrounding an enemy base with sandbags and blocking the enemy inside. Developers intentionally chose to not let the AI target sandbags, to prevent its attacks on an enemy base being bogged down in clearing sandbags.

[aaatoysandmore]

Interesting Mobeer very informative.

[vonRocko]

I can think of several games that were released without anyone ever playing a complete game. Pride of nations and War in the east are just two examples. They didn't even know if the game can be played to the finish!

[DSWargamer]

In my youth, a new wargame would be said to be 'coming' measured in years.

When it was released, it might get a few new editions of the manual to correct glaring issues.

Today I think the market expectations are part of the problem and the speed required to bring to market likely the rest of the problem.

How does one seriously play test a game that inherently takes months and months to play through ONCE, and have it ready in under a year?

The more complicated and the more detailed and over all complex a game needs to be, the more likely no one will be able to uncover all the seemingly obvious glitches once it is released to hundreds or more fans.

I have numerous wargames which I purchased early on and then life happens and I just don't get around to playing the game, and then I find myself playing it, and it had seen a score of patches. Sometimes it simply doesn't pay to expect the world from something that simply can't realistically be accomplished in a short time frame. People complain about being paid beta testers. Ok, fine enough complaint. The alternative, is you get a new wargame maybe every 3 years all publishers and developers inclusive.

[CapnDarwin]

As a dev currently supporting and working a Matrix title let me throw out a few items for discussion based on the OP's comments and other in the thread.

1. There is a world or difference between a design studio working 24/7 and a small group like ours working evening and weekends between real jobs, families, etc. With that being said, we have tried our best to release the most bug free game we can. The reality is there is no way to do that. Hell the major developers like EA can't do it with large devoted teams. The reason is there are just too many hardware, OS, and driver combos to test. Throw international localization issues and stupid things like 150% font scaling in Win8 (thanks MS for that little UI breaking gem) and it is a madhouse. We did our best to cover all bases and we missed some of those items. We are working to fix them and most good dev teams will.

2. Do we play our own game. Yes. Not as much as testers and players of course because we have to do code, data, art, sound, docs...you get the idea. In a lot of cases we will run small tests to see if fixes are working. In our case we have a really good QA Czar with Charles and Steve as he does scenarios and tests them. Along with testers and players we get fed bug reports and suggestions that we then work on. That cycle leaves little time for sitting down and running through a campaign. Another point that is critical here is "fresh eyes". When your are waist deep in the code and mechanics you can miss broken things because you are so focused on the task at hand. Having those beta testers and relentless eagle eyed QA folk look at the game from angles you just don't see from the debugger.

3. I really think the indy/small team games end up being better and less buggy because we have a vested interest and love of this niche market. I'm not going to retire to Hawaii any time soon on what the game makes. Large companies are all about schedules and profits. You ship on X day come hell or high water in those environments. It does not surprise me one bit what Mobeer commented on. That's big business right there. Way too many examples of that kind of company out there. Matrix does a good job of having its developer's support the game long after release. Rare. So embrace it.

4. AI rating. Wow there's a hornets nest. How do you define AI? What would be logic parameters to measure by? I think our AI is on of the best out there right now. It's not perfect in some areas, but it doesn't cheat and it is dynamic enough to play the same scenario differently based on what the player is doing. I agree that seeing some form of measure of AI on games would be cool. Just needs to be done right to reflect just what the AI is doing and capable of.

Just some long winded comments after a long day of debugging, feature tweaking, and forum support.

Thanks for listening. [8D]

[aaatoysandmore]

Thanks for replyin Capn. All info is welcome. [:)]

[Fred98]

I have been a tester for a few wargames.

A wargame might have a few scenarios and a (say) 80 turn campaign.

Testers are not given instructions and so will usually play the 80 turn campaign.

By turn 10 or 15 the next version is released and we start all over again.

As for the scenarios only a few will be tested and never to the end as there is never enough time. Partly because the next beta is released and partly because testers have a real job too.

Testers need to be paid. In a turn based game they might be paid by the turn.
.
.

[british exil]

ORIGINAL: Capn Darwin

As a dev currently supporting and working a Matrix title let me throw out a few items for discussion based on the OP's comments and other in the thread.

1. There is a world or difference between a design studio working 24/7 and a small group like ours working evening and weekends between real jobs, families, etc. With that being said, we have tried our best to release the most bug free game we can. The reality is there is no way to do that. Hell the major developers like EA can't do it with large devoted teams. The reason is there are just too many hardware, OS, and driver combos to test. Throw international localization issues and stupid things like 150% font scaling in Win8 (thanks MS for that little UI breaking gem) and it is a madhouse. We did our best to cover all bases and we missed some of those items. We are working to fix them and most good dev teams will.

2. Do we play our own game. Yes. Not as much as testers and players of course because we have to do code, data, art, sound, docs...you get the idea. In a lot of cases we will run small tests to see if fixes are working. In our case we have a really good QA Czar with Charles and Steve as he does scenarios and tests them. Along with testers and players we get fed bug reports and suggestions that we then work on. That cycle leaves little time for sitting down and running through a campaign. Another point that is critical here is "fresh eyes". When your are waist deep in the code and mechanics you can miss broken things because you are so focused on the task at hand. Having those beta testers and relentless eagle eyed QA folk look at the game from angles you just don't see from the debugger.

3. I really think the indy/small team games end up being better and less buggy because we have a vested interest and love of this niche market. I'm not going to retire to Hawaii any time soon on what the game makes. Large companies are all about schedules and profits. You ship on X day come hell or high water in those environments. It does not surprise me one bit what Mobeer commented on. That's big business right there. Way too many examples of that kind of company out there. Matrix does a good job of having its developer's support the game long after release. Rare. So embrace it.

4. AI rating. Wow there's a hornets nest. How do you define AI? What would be logic parameters to measure by? I think our AI is on of the best out there right now. It's not perfect in some areas, but it doesn't cheat and it is dynamic enough to play the same scenario differently based on what the player is doing. I agree that seeing some form of measure of AI on games would be cool. Just needs to be done right to reflect just what the AI is doing and capable of.

Just some long winded comments after a long day of debugging, feature tweaking, and forum support.

Thanks for listening. [8D]

I agree with Capn here. I bought a game he had worked on release day, why? Because I had read what the beta testers had posted. I knew that the game would have some faults, but I wanted that game ASAP.

Now go to the forums read the sections where games are in development, there are always people who want the devs to release their game straight away, no matter how many bugs the game may still have, no matter what the price.

When my afore mentioned game was released, threads were made where bugs were, what typos had been made in briefings etc.. The devs had already mentioned that some problems/mistakes/bugs were still in the game, but to be able to release the game certain things had to remain to be fixed at a later date/patch.

The forum created a wishlist, what they would like to see in the game. Certain points were included in later patches.
Certain aspects of the game were buggy, promises were made to try to change them. Not just a click of a key, as so many other items crop up when changing codes.

The devs and testers of this game read the threads and answer them, somtimes within minutes.

Perfect game release, no. Perfect help and service of the team, yes.
Are they sitting back patting each other on the shoulder, sitting back no, they are developing their game further. Although they deserve a pat on their backs.


I have tried as a beta tester, it is a lot of work. You play a turn, find bugs, save the game, make screenshots. After next update play the same turn, see any differences, next turn till the next update and the turn continues.

All of this after a days work, real life, family etc. Payment? Well I got the game free of charge and was mentioned in the credits. I feel that I did not contribute that much to the final product. But it makes me enjoy a new game, even if I do find bugs.

A game can't be released without any bugs. It would take too much time developing and small teams would reach a burnout point, their families may reach breaking points.
And honestly do you want to wait years for a release,after hearing a certain game is in development or to read in the forums the release date has been postponed again and again?


I know the big companies release games that are so bad to play that it is impossible to enjoy the game. I have had quite a few, where I have just stopped playing them and removed them from my harddrive.
These companies charge a full price for a product that is not fully developed. The people that have bought the games should get a complete refund. But that will never happen.

A certain city building game was so bad that people refused to buy the game, I bet the uproar caused a few heads to roll.

All in all, I am a happy gamer here at matrix, most games are bug free and update patches are always free of charge.

Thanky matrix.

Mat

[Jeffrey H.]

I don't think dev's really play them from a "true" players perspective. This creates what I call "blind spots" in their views of a game.

These blind spots are often identified but are essentially ignored for whatever reason. I think often there is an unwillingness to admit anything might be wrong/missing/fixable for fear of a smear campaign developing or delaying the title release.

In this sense they are not "players" per se, they are herding a small team towards a goal. Sometimes the results are good, sometimes not.

They also don't seem to want to admit it when they have made up their minds on something. this compromises trust, but they don't understand this or don't care.

Ive been a pre-alpha, alpha and beta tester for games and scenarios. Usually it's an enjoyable experience but I do notice that they seem to have some serious issues when it comes to discussing openly some aspects of the game.

[CGGrognard]

No, for the most part I don't believe developers play the games they work on. Consider most Formula One mechanics don't actually drive the cars they work and they don't necessarily know how the tweaks to the engine and onboard computers actually work out until the car is out on the track. And that's just it. "Real world" testing still beats computer testing, especially for software since a software test program will be built around the "known" issues and concerns. What's needed are real life people (testers) trying to break the program to identify some potential failures of the software.

I'm not a developer, but it seems to me as in earlier posts, they have quite a lot on their plate. I appreciate the "small Indy" type developers for their calling is more of a passion than totally monetary gains.

[Fred98]

Yes I agree. Testers will always play a normal turn. Testers will never play a weird turn.

Testers should be trying to break the game.

.

[aaatoysandmore]

ORIGINAL: Joe 98

Yes I agree. Testers will always play a normal turn. Testers will never play a weird turn.

Testers should be trying to break the game.

.

I agree and that is what I always tried to do with SSI games I tested. Thus you didn't see bugs and flaws of games I tested. That was back in the day of 3-1/2 disks too that I had to install each week. I always tried to do things that weren't suppose to be done.

[Jamm]

While working on the 'Ghost Divisions' expansion with WCS, I played through a lot of scenarios with both Eric and Erik.
The games were a lot of fun, they're both good players btw.
As well scenarios and campaigns are played over and over again, looking for just the right balance.