Playtesters?

Carriers At War is Strategic Studies Group famed simulation of Fleet Carrier Air and Naval Operations in the Pacific from 1941 - 1945.

Moderators: Gregor_SSG, alexs

User avatar
e_barkmann
Posts: 1292
Joined: Tue Apr 18, 2000 8:00 am
Location: Adelaide, Australia

RE: Playtesters?

Post by e_barkmann »

just the host, so a saved game will be transferred from host to client upon loadup, and pretty sure this applies to scenarios where a mis match occurs too.

cheers
Scourge of War multiplayer group

http://steamcommunity.com/groups/sowwaterloo
GoodGuy
Posts: 1506
Joined: Wed May 17, 2006 5:36 pm
Location: Cologne, Germany

RE: Playtesters?

Post by GoodGuy »

ORIGINAL: Prince of Eckmühl

...but the fact of the matter remains that the developer and publisher are responsible for the quality of the product that we purchased, not the play-testers.
Oh yes, my "pleading" wasn't meant as bashing of play-testers. The testers are supposed to be led/managed by the dev, and it's the dev who has to direct things, besides the testers reporting about findings additionally/along the way. I do work in the localization business, so I know what/why things can go wrong.
...... because they are as grossly overworked as they are woefully under-compensated.
yepyep. In this niche-market, it's even mostly about "0" compensation, besides getting a copy of the release version.
As for my suggestion that a patch might have been in the works prior to the game's release, it is but one of myriad possibilities which might explain...
I know. [:)] I just don't accept this particular reason. [:D]

Btw, my post above was meant to be a general statement, not aimed at SSG in particular, but at the software industry in general. Keep it up POE :P
"Aw Nuts"
General Anthony McAuliffe
December 22nd, 1944
Bastogne

---
"I've always felt that the AA (Alied Assault engine) had the potential to be [....] big."
Tim Stone
8th of August, 2006
User avatar
Prince of Eckmühl
Posts: 2459
Joined: Sun Jun 25, 2006 4:37 pm
Location: Texas

RE: Playtesters?

Post by Prince of Eckmühl »

ORIGINAL: Chris Merchant

just the host, so a saved game will be transferred from host to client upon loadup, and pretty sure this applies to scenarios where a mis match occurs too.

cheers

At the risk of deepening GoodGuy's suspicions about my wiring, [;)] I'm gonna ask if you could elaborate on the meaning of the term "mis match" and any impact that it might have on the ensuing game.

PoE (aka ivanmoe)
Government is the opiate of the masses.
MarkShot
Posts: 7539
Joined: Sat Mar 29, 2003 6:04 am

RE: Playtesters?

Post by MarkShot »

I have stated elsewhere that I think game play and AI of games produced in the early 1990s was generally superior to what passes for the same in the late 2000s.

Another thing which I firmly believe is that software quality (for all types of software not just games) of games has suffered greatly as the Internet became established on a new medium within society.

The Internet has contributed greatly to our hobby:

(1) Small independent developers can reach their niche market to sell their product where in the past they would have never had a hope of finding a publisher or shelf space.

(2) Communities of like of common interested players can form across international borders, regions, and other demographics.

(3) Modders and scenario designers can produce content for game engines that effectively give them life way beyond what the original development team/publisher intended.

(4) The development of point #3 along with cheaper CPU cycles, memory, and disk storage has resulted in the data structures of many games being moved out of the EXE to separate files. And in many cases, these files are plain text files. The games of the early 1990s saw more of the core game hardcoded in the EXE and data files even when used where usually tightly encoded binary data that required a great deal more skills to manipulate than NOTEPAD.

But the Internet has at same time hurt our hobby. Software quality has degraded and continues to do so:

(1) Product cycles were reduced and competition was increased. All of a sudden everyone is working on an ACW game. First, let's face it there are only so many historical epics worthy of mass marketting. If one party senses money to be made, then they all do. The race begins to be out the door first and hit Q3/Q4 where the money is. The difference between first and second place can be the difference between moving on to the next game or financial insolvency. So, the pressure is very real. Good companies will have experience project management and know how to cut features such that they make their deadlines while still delivering a worthwhile project. Poor companies will be overwhelmed and cut corners in a haphazard fashion and release a can of worms. Customers will be burned. There will be a bad taste. If customers are lucky, Q1 and Q2 will be used to fix the game and the rest of the less serious issues will be addressed by the modders.

Microsoft Windows and the Internet started a trend in the software industry of accelerated project schedules. Projects moved from carefully planned giants that were build on computers costing millions to things hacked together on Monday to show users and investors on Friday. The trend really started when Microsoft released Windows 3.0 and Visual Basic 1.0. This began it for those of us who worked in the industry. In 1995, when the Web overshadowed the release of Windows 95, the Internet brought accelerated software development schedules through quick visual mock-ups to the business world at large and not just IT departments.

I've said a lot. But the basic point is that the gaming industry suffers the same pressures that any other forms of software development suffer.

(2) The Internet has made code defects (bugs) appear to be very cheap and inexpensive. In 1987, getting fixes out to customers was a very expensive proposition. It involved distributions of tapes and other physical media. It was hugely expensive and much more cost effective to simply kill the bugs before shipping your product. By 1997, the major vendors all had FTP servers and it had become cheap to just throw patches up on the servers and let customers retrieve them. (Of course, this began before the Internet with the first BBS modem services.) The intense discipline of testing and quality assurance faded away during the 1990s, since the penalties and pressures which created that culture dissappeared.

So, it was with the gaming industry. In the early 1990s, a game with major bugs got bad press. It could not be easily fixed. The product would not move and money and time was lost. Minor bugs and enhancements tended to be addressed through expansion packs that were both patches and additional content add-ons. However, in the late 1990s and problems with a game could be fixed by the customers patching themselves at any time. Zero cost for bugs (so no pressure enforcing a culture of quality) while (see above) lots of pressure forcing fast and slopy work to market. As has been noted in this thread, it has become standard practice simply release unworkable code as version 1.00 and fast track a patch waiting for the game to reach the store shelves.

Clear and simple, it is the Internet which has created this situation. However, as I have demonstrated, there are also wonderful gains which have come about via the Internet too. Also, it is not only the developers/publishers who are to blame for atrocious software quality. We, the customers, are to blame as well. We regularly pre-order games, download them immediately, drive to stores to get them, ... We purchase these products before waiting to see if they are solid. We just assume that any critical problems will be fixed and we are entitled to these fixes. If we did not behave as we do, those who produce games would not behave as they do. So, our buying patterns are also very much to blame for this. At best, reputable companies ultimately fix their products. Other companies, simply abandon them.

This explains why I just got a new PC, but for the most part every game on my PC with one or two exceptions was published at least two years earlier. By now, the game is either worth having or it is not. In 2009, I will probably be purchasing quite a few titles from 2007. :)

As long as I am handing out blame, then the gaming community review web sites are to blame too. Very few of them will really expose a game as unplayable pile of CTDs and bugs. Very few of them will bury a product for being a cinch to beat after only a few days worth of play experience. So many of these sites are whores who are in bed with the products they are responsible for objectively reviewing. They are not consumer advocates, but in fact just extensions of the marketting apparatus of the industry. Look at some games which are a few years old that are widely known to be only mediocre. Now, go back and look at the reviews which they games received a few years back. For the vast majority, the review will relect a much more upbeat assesment than the product actually deserved.

---

I realize that some of what I have said here may not be in all the best interests of increased sales for my friends at Matrix. However, I was trying to paint a picture of an industry and history in broadstrokes. This not meant as a criticism of any one party, but once again an attempt to talk of historical trends and underlying causes that has shaped these events.
2021 - Resigned in writing as a 20+ year Matrix Beta and never looked back ...
User avatar
e_barkmann
Posts: 1292
Joined: Tue Apr 18, 2000 8:00 am
Location: Adelaide, Australia

RE: Playtesters?

Post by e_barkmann »

At the risk of deepening GoodGuy's suspicions about my wiring, I'm gonna ask if you could elaborate on the meaning of the term "mis match" and any impact that it might have on the ensuing game.

Alex could elaborate here, but the game routines check the integrity of scenario data (and game version) prior to commencing the game on both machines, to eliminate crashes due to bad data mismatches.

cheers
Scourge of War multiplayer group

http://steamcommunity.com/groups/sowwaterloo
GoodGuy
Posts: 1506
Joined: Wed May 17, 2006 5:36 pm
Location: Cologne, Germany

RE: Playtesters?

Post by GoodGuy »

ORIGINAL: MarkShot

In 1987, getting fixes out to customers was a very expensive proposition. It involved distributions of tapes and other physical media. It was hugely expensive and much more cost effective to simply kill the bugs before shipping your product.
Well, in fact, these type of fixes were reserved to commercial customers/companies, where big amounts of money and/or possible claims for recourse were involved. These fixes did not reach / involve the end-user. Somewhere between 1989/1991 German and British Amiga magazines (these covered software applications, OS's etc, not games) started to release fixes on their cover (floppy!!!) disks, most notably for programming tools and audio/DTP software, as these programs were on a way higher level than games, price-wise. So this procedure started even earlier than some may think (= yrs before the internet was widely used/accessible). Given, fixes were rare and rarely needed.

So many of these sites are whores who are in bed with the products they are responsible for objectively reviewing. They are not consumer advocates, but in fact just extensions of the marketting apparatus of the industry. Look at some games which are a few years old that are widely known to be only mediocre. Now, go back and look at the reviews which they games received a few years back. For the vast majority, the review will relect a much more upbeat assesment than the product actually deserved.

Well, it's even worse with some game mags (print). There was a scandal here in Germany (this year or last year, can't remember), where an online mag thought it should mutate to some investigative magazine, publishing a letter or fax that seemed to prove that some print mag received bucks (and additional orders for expensive ads in that same mag) from a major software company for a good review of a major title. While that online mag failed to come up with more supporting evidence, I am pretty sure that there is some "match fixing" in the industry, if I'm reading through certain reviews.
"Aw Nuts"
General Anthony McAuliffe
December 22nd, 1944
Bastogne

---
"I've always felt that the AA (Alied Assault engine) had the potential to be [....] big."
Tim Stone
8th of August, 2006
User avatar
Prince of Eckmühl
Posts: 2459
Joined: Sun Jun 25, 2006 4:37 pm
Location: Texas

RE: Playtesters?

Post by Prince of Eckmühl »

ORIGINAL: MarkShot

So many of these sites are whores who are in bed with the products they are responsible for objectively reviewing. They are not consumer advocates, but in fact just extensions of the marketting apparatus of the industry.

One of the least understood practices within the gaming media is the renting out of reviewers to help market a publisher's games. It's a straight-up, fee-for-service relationship. I'm aware of at least one quid pro quo arrangement in which a "journalist" was offered a stake in a fledgling development house in return for helping market its first game. When I read some of the stuff that's written about computer games, I can't help but be reminded of the old Jefferson adage, "The most truthful part of the newspapers is the advertisements." It would be more respectful of the readership were these publications to simply label them as such.

Let's go find some bugs!!!

PoE (aka ivanmoe)
Government is the opiate of the masses.
MarkShot
Posts: 7539
Joined: Sat Mar 29, 2003 6:04 am

RE: Playtesters?

Post by MarkShot »

Moe,

Where do I sign up for such work? I could use some extra money as I just order 25 ppm single pass color laser printer network ready with duplex for my business today. Of course, it will also do a splendid job at printing out color PDF game manuals which are getting common place these days. I am getting on in years and the ole all-in-one color inkjet has been driving me towards senility! :)
2021 - Resigned in writing as a 20+ year Matrix Beta and never looked back ...
GoodGuy
Posts: 1506
Joined: Wed May 17, 2006 5:36 pm
Location: Cologne, Germany

RE: Playtesters?

Post by GoodGuy »

ORIGINAL: Prince of Eckmühl

Let's go find some bugs!!!

No, I don't wanna find any bugs. I want a bug-free environment. *GG stomping on the ground like a lil kid*

[:D]
"Aw Nuts"
General Anthony McAuliffe
December 22nd, 1944
Bastogne

---
"I've always felt that the AA (Alied Assault engine) had the potential to be [....] big."
Tim Stone
8th of August, 2006
User avatar
Prince of Eckmühl
Posts: 2459
Joined: Sun Jun 25, 2006 4:37 pm
Location: Texas

RE: Playtesters?

Post by Prince of Eckmühl »

ORIGINAL: MarkShot

Moe,

Where do I sign up for such work? I could use some extra money as I just order 25 ppm single pass color laser printer network ready with duplex for my business today. Of course, it will also do a splendid job at printing out color PDF game manuals which are getting common place these days. I am getting on in years and the ole all-in-one color inkjet has been driving me towards senility! :)

Speaking as someone who wants to see you leave this world emotionally whole and as genuine a person as you seem, I'd suggest sticking with you're daytime job. There's something heeling about having endured an honest-day's work, at the end of which you can look at the people that you love and know that the food on the table is there because your customers got their money's worth.

PoE (aka ivanmoe)
Government is the opiate of the masses.
MarkShot
Posts: 7539
Joined: Sat Mar 29, 2003 6:04 am

RE: Playtesters?

Post by MarkShot »

Very funny!

The stories I can tell about the gaming industry is nothing compared to those I have about trading floors, pharmceutical companies, commercial banks, big accounting firms, software consultancies, Internet startups, VC funding, dot-coms, ... If you were to hear some of those stories, then you would know that the PC game industry represents the very highest ideals of integrity and delivering value to the customer; at least, in relative sense. You got to be big to steal big. The gaming industry is too small potatoes to take seriously.
2021 - Resigned in writing as a 20+ year Matrix Beta and never looked back ...
MarkShot
Posts: 7539
Joined: Sat Mar 29, 2003 6:04 am

RE: Playtesters?

Post by MarkShot »

By the way, I agree that we should all get back to playing CAW or whatever. This thread has gotten much too serious for a group of middle aged men clicking on little toy ships on a map of the Pacific! :)
2021 - Resigned in writing as a 20+ year Matrix Beta and never looked back ...
User avatar
Prince of Eckmühl
Posts: 2459
Joined: Sun Jun 25, 2006 4:37 pm
Location: Texas

RE: Playtesters?

Post by Prince of Eckmühl »

ORIGINAL: MarkShot

By the way, I agree that we should all get back to playing CAW or whatever. This thread has gotten much too serious for a group of middle aged men clicking on little toy ships on a map of the Pacific! :)

Agreed!!!

PoE (aka ivanmoe)
Government is the opiate of the masses.
User avatar
JD Walter
Posts: 235
Joined: Fri Jun 20, 2003 7:26 am
Location: Out of the Silent Planet

RE: Playtesters?

Post by JD Walter »

* ...Ahem...*
 
If I may beg the Gentlemen's indulgence (before returning to our regularly scheduled game of CAW), I would like to add the following in this regard...
 
As long as I am handing out blame, then the gaming community review web sites are to blame too. Very few of them will really expose a game as unplayable pile of CTDs and bugs. Very few of them will bury a product for being a cinch to beat after only a few days worth of play experience. So many of these sites are whores who are in bed with the products they are responsible for objectively reviewing. They are not consumer advocates, but in fact just extensions of the marketting apparatus of the industry. Look at some games which are a few years old that are widely known to be only mediocre. Now, go back and look at the reviews which they games received a few years back. For the vast majority, the review will relect a much more upbeat assesment than the product actually deserved.
 
I have a friend who presently works in the game review magazine industry. She recently changed publications. After this move, she relayed to me that, at her previous publication, due to the pressing need to be the first to print with a "timely" feature on whatever was released that month, the majority of reviews were based on a single playing of the game at the easiest possible setting. This was generally completed in one day, then used as the basis for a 250-300 word "review".
 
Only the most egregious faults were ever seen, much less written down and reported on. She remarked that in her own personal experience, only a CTD or BSOD would warrant a note that the game had a "bug".
 
After this, it was thrown on the shelf (or, more accurately, into a pile on the floor of a spare office), and it was back to WOW for the rest of the workweek.
 
Employees who were given two such writing assignments were considered overworked. They would complain loudly about losing 16 man-hours levelling up their character(s) in WOW, and falling behind the rest of the office.
 
This trend got so bad, some reviewers started quoting from internet sites instead of playing the game thoroughly themselves, in order to catch things they might have missed in their cursory look at it for the article, so that they could be sure to note something in their write-up that was common knowledge on the 'Net, and not be embarrassed at missing an obvious flaw that was becoming too well-known and talked about in the "early adopter" gaming community.
 
Moral: Readers who subscribe to such magazines should be aware that many print reviews are based on, at most, an eight-hour session with the game, at the easiest setting allowing the ending to be played. Be forewarned.
 
We may now return to our appointed forum ...
MarkShot
Posts: 7539
Joined: Sat Mar 29, 2003 6:04 am

RE: Playtesters?

Post by MarkShot »

Time to go back to playing. Look at this disaster which has just befallen me at Wake! :(

Image
Attachments
ouch.jpg
ouch.jpg (27.06 KiB) Viewed 396 times
2021 - Resigned in writing as a 20+ year Matrix Beta and never looked back ...
User avatar
Fred98
Posts: 4019
Joined: Fri Jan 05, 2001 10:00 am
Location: Wollondilly, Sydney

RE: Playtesters?

Post by Fred98 »

I was not a tester on this game.

But, the other thing about testing, it ‘s usually impossible to complete a scenario H2H because the developers release the next beta version before you get too many turns in.

As a result only the first 6 or 8 or 10 or so turns of any scenario get tested H2H so its difficult to check the balance of longer scenarios.

-
User avatar
Oleg Mastruko
Posts: 4534
Joined: Sat Oct 21, 2000 8:00 am

RE: Playtesters?

Post by Oleg Mastruko »

ORIGINAL: Joe 98

But, the other thing about testing, it ‘s usually impossible to complete a scenario H2H because the developers release the next beta version before you get too many turns in.

As a result only the first 6 or 8 or 10 or so turns of any scenario get tested H2H so its difficult to check the balance of longer scenarios.

Very very true, but not for this game [:D]
User avatar
Oleg Mastruko
Posts: 4534
Joined: Sat Oct 21, 2000 8:00 am

RE: Playtesters?

Post by Oleg Mastruko »

ORIGINAL: MarkShot

Time to go back to playing. Look at this disaster which has just befallen me at Wake! :(

Image

Don't know about beta testers (after all it's the thread about them [;)]) but developers don't think it's a disaster at all! Of course I am talking about CAW 2007, not CCAW. For more details see my thread "Ridicolous scenario design" (should be on a page 2 or 3 by now).
User avatar
Duck Doc
Posts: 740
Joined: Wed Jun 09, 2004 12:22 am

RE: Playtesters?

Post by Duck Doc »

You are kidding me, right?

Let's see: we have found the one area of human concern magically sequestered from the foibles of human nature [;)]. Everybody checks their character defects at the door when beta testing - or developing games. No brown nosing? No fragile egos? No favoritism? Come on. Who are you trying to kid? (I'm guilty even & I ddn't even test in the real sense.)

This is the real world. All is corruption. From the top of the govenment down to the guy who mows your lawn. Your senator & congress person, your hairdresser.

Besides the testers are confined to the universe the developers create. There are constraints on criticism. Check out the difference between how the ciriticism is recieved on the various newly released games on the forums here.

Now, nobody is making a killing making wargames we old geezers like to play. Let's not make a federal case out of these things. Let's lighten up a bit. Let's cut everybody a bit of slack: the developers, the beta testers & the players. Let's have some fun...

...but let's not go on applauding the emperor's new clothes.

My problem is I can't keep my mouth shut. I never did & never will. Too much integrity for that. I'll never learn. Hard to get ahead this way...


.. but I digress [:D].

ORIGINAL: MarkShot

Very funny!

... then you would know that the PC game industry represents the very highest ideals of integrity and delivering value to the customer; at least, in relative sense. You got to be big to steal big. The gaming industry is too small potatoes to take seriously.
User avatar
Prince of Eckmühl
Posts: 2459
Joined: Sun Jun 25, 2006 4:37 pm
Location: Texas

RE: Playtesters?

Post by Prince of Eckmühl »

ORIGINAL: Joe 98

I was not a tester on this game.

But, the other thing about testing, it ‘s usually impossible to complete a scenario H2H because the developers release the next beta version before you get too many turns in.

As a result only the first 6 or 8 or 10 or so turns of any scenario get tested H2H so its difficult to check the balance of longer scenarios.

-

There was a game that I tested that had a scenario that was three to five hours in length depending on the variant that one might draw. The maximum time compression was 3-1, so I was looking at least an hour of testing to reach the conclusion of the game. Toward the end of play-test cycle, a bit of a crisis arose because the testers were reporting the game ending without the scenario results being displayed.

Anyway, While everyone and his uncle reported to their buddies for a Sunday's afternoon of NFL football, I tracked that bugger down and uploaded a saved game clocked to three seconds before the failure, and the issue was quickly resolved. That's playtesting.

Even using Run5, I could have done the same with CaW-007 in a TINY fraction of the time that I had devoted to the bug in the other game, the key being the ability to accelerate the game clock. If I've been short with Gregor at times, that's why. He or his crew ought to be able to reproduce these bugs, more or less effortlessly, with a minimum of time and effort expended. That their customers are asked to do it, I believe, is unacceptable.

PoE (aka ivanmoe)
Government is the opiate of the masses.
Post Reply

Return to “Carriers At War”