How to be a good tester?

SPWaW is a tactical squad-level World War II game on single platoon or up to an entire battalion through Europe and the Pacific (1939 to 1945).

Moderator: MOD_SPWaW

User avatar
Resisti
Posts: 1236
Joined: Mon Jan 22, 2001 10:00 am
Location: Livorno, Italy

How to be a good tester?

Post by Resisti »

A lot of threads on how to be a good scenario designer (thanks WB,btw!),but none on this matter.
I know maybe this is more subjective than design methods, but nevertheless I'd be HIGHLY interested in hearing from some of the top (and not only) designers which kind of feedback they expect,when sendng out a scenario for testing.
Maybe a sort of "rules list",altough necessarily a bit generic and subject to personal interpretations,would be useful; well it would be for me at least :)

Cmon WB,Brent,Redleg,Marauder,Paul V.,and all the other design gurus, share yr toughts with us.

Naturally, opinions from top testers(Warrior? Alexandra?,who else?) as well as anybody else are welcome!
Federico "Resisti" Doveri
User avatar
Warrior
Posts: 1648
Joined: Thu Nov 02, 2000 10:00 am
Location: West Palm Beach, FL USA

Post by Warrior »

First and foremost, I look for technical glitches: hidden buildings, vehicle "traps" not noted on the map, unattainable victory hexes (perhaps set on a cliff or some such where it's impossible to get to them), unit anomalies (an ammo dump with ATG's, weird weapons, etc.), and so on. I also look at the text for spelling and grammar errors.

Then I try to determine if the scenario is historically accurate. I'm not as "up to speed" on this as I could be, so I generally trust the designer. I get a "feel" for the play - is it too easy, too hard, does it leave me with a feeling of identification with the units involved, etc. Did I like the scenario, did it make me crazy, would it be possible to win, was it crap? If the designer has asked for my opinion, I give it, otherwise I stick to the technical stuff.

I give a brief AAR so the designer will know how I approached the tactical problems presented by the scenario, i.e., did I attack from the north or south, did I split my forces, was maneuver a big part of it, etc.

There are other "intangible" factors, of course, but this is generally what I report. If a designer has follow-up questions or needs clarification, I address those.

Oh, and if it's one of Sapper Don's scenarios... I naturally tell him it sucked no matter what.
:D :D

[ July 02, 2001: Message edited by: Warrior ]
Retreat is NOT an option.

Image
User avatar
Resisti
Posts: 1236
Joined: Mon Jan 22, 2001 10:00 am
Location: Livorno, Italy

Post by Resisti »

BUMP

Hmmm,it seems all designers disappeared....

:(
Federico "Resisti" Doveri
Larry Holt
Posts: 1644
Joined: Fri Mar 31, 2000 10:00 am
Location: Atlanta, GA 30068

Post by Larry Holt »

I do a lot of software testing, here is what I consider important:

Be able to report real findings, that is what happened, not what you remember happening. For this, turn on logging (alt-L) and keep a paper or electronic journal. You can alt-tab the game into the background to give yourself time to take notes.

Have a plan, what are you going to be looking for, balance, movement pathing, etc.? That way you don't waste time, you finish the scenario then you wonder about something and have to replay it.

The designer should give you a list of changes if this is itterative testing. Verify them.
Never take counsel of your fears.
Mark Ezra
Posts: 258
Joined: Fri Dec 29, 2000 10:00 am
Location: Jasmin Ranch, Acton CA

Post by Mark Ezra »

Being a good Tester? Objectivity is important. If one gets caught up in the game objectivity suffers. If one likes a particular designers work (or not) it becomes very important to remain impartial. As a tester I look first at the game description. Does it give time date length and weather. Does it describe events clearly. Does it egage the reader? Finally is the lay out, punctuation and spelling (my big challange) OK. Next I review the map. Does the map use the entire surface well. Is it functional as well as interesting in both the over view and detailed view. Are roads free of hidden "stone Bldg?) are there other defects common to the map editor (like a #3 hill hex left over in your city or wheat field). Finallly can the AI move reasonable well or is it hindered by too much rough or soggy terrain. If it is a historical scen, does the map resemble the actual battlefield. Now go to the OOBs Are the OOB's for each side reasonable to the scen. If it's historical does it match known OOB's. If it's not does it have a intersting blend of units. And now we test the game!...Does it play through the lengtgh of the game? was it too easy, too hard, to silly? Did it create intensity, excitement or was it just an exercise. Did the AI act as the designer wished? If it's historical does it give the flavor of the actual event?And finally the score. How'd I do, It's always good to enclose a screenshot of the score. It will help the designer to understand the Vhex cost, casualties, units activity and is a clear proof that the entire game was played. If for some reason the game was not completed the designer needs to know. Now all that needs to be done is report the findings and recomendations in a clear logical step by step manner.
All Hail Marx and Lennon
User avatar
Warrior
Posts: 1648
Joined: Thu Nov 02, 2000 10:00 am
Location: West Palm Beach, FL USA

Post by Warrior »

Originally posted by Mark Ezra:
Objectivity is important. If one gets caught up in the game objectivity suffers... Now all that needs to be done is report the findings and recomendations in a clear logical step by step manner.
Objective? Clear? Logical?? My highest praise for a scenario goes to the one that makes me scream profanities at the monitor and pound on the keyboard & mouse! :D ;) :D

[ July 02, 2001: Message edited by: Warrior ]
Retreat is NOT an option.

Image
Mark Ezra
Posts: 258
Joined: Fri Dec 29, 2000 10:00 am
Location: Jasmin Ranch, Acton CA

Post by Mark Ezra »

Warrior: That's cause you're a GOOD player...LOL I'm lucky to get a draw!
All Hail Marx and Lennon
User avatar
Grenadier
Posts: 658
Joined: Wed May 10, 2000 8:00 am
Location: Newport Beach, CA USA

Post by Grenadier »

Most of the time I just want to know if the scenarios are being played :D

Seriously,

I look for playability of the scenario. Is it predictable? Is it accurate? The ones I like the least are the "go for the first VO's and defend until the AI is bled dry then take the rmaining VO's". The best cure for this are the time per turn VO.

The feedback that helps me the most is finding out if the scenario played out historically or was the player able to change an historic outcome as most of my scenarios are designed to be recreations of actual battles, such as Seelowe and Villers-Bocage. I want to know if the VO's worked as designed, if the scenario was fun to play, if it was realistic, was the balance of forces OK? Were there any surprises?

Any kind of feedback is welcomed by me. That way I know they are being played.
Brent Grenadier Richards




__________________
Image

[url=http://
User avatar
Don Doom
Posts: 1984
Joined: Sat Sep 23, 2000 8:00 am
Location: Lost somewhere in the upper backwoods of Michigan!

Post by Don Doom »

I agree with DGPT[Brent], I generally look to see if the tester found it playable, if the v-hexes where reachable in the time allowed, if the map was historicly correct as possible. If the .txt file gave enough info for the battle without giving it away. The two forces are balenced.
Don

ps Warrior who is sapper don?
Doom
Vet of the Russian General Winter
For death is only the begining
User avatar
Warrior
Posts: 1648
Joined: Thu Nov 02, 2000 10:00 am
Location: West Palm Beach, FL USA

Post by Warrior »

Originally posted by Don Doom:
ps Warrior who is sapper don?
Don "Sapper" Llewellyn, coordinator of the SPWaW Tactical Training Center and all-round good guy.
Retreat is NOT an option.

Image
User avatar
Warrior
Posts: 1648
Joined: Thu Nov 02, 2000 10:00 am
Location: West Palm Beach, FL USA

Post by Warrior »

Originally posted by Mark Ezra:
Warrior: That's cause you're a GOOD player...LOL I'm lucky to get a draw!
Mark, don't worry, you're a good player. The evil, nasty designers are the ones to blame for all the draws - it's not your fault. They get a perverted, degenerate satisfaction out of making scenarios that are difficult in the extreme. :D :D :D
Retreat is NOT an option.

Image
Don
Posts: 662
Joined: Wed Jul 12, 2000 8:00 am
Location: Elk Grove, CA (near Sacramento)

Post by Don »

Ah, good old Warrior! We worked together so much on the MC that sometimes while positioning extra Matida's for a flank attack I'd be thinking of Steve's reaction! And wishing I could be a "fly on the wall" when they came in! :)

The worst kind of testing result are the ones that simply say "I like it" or "I don't like it". These pretty much tell you nothing, so the review has to be broken down into different areas such as map, objective placement, etc. WB was the first one I saw to ask for specific things from testers, and that carried over into the MC. This way you get more consistent reports, and feedback that you can really use to make the scenario better.
Don "Sapper" Llewellyn
User avatar
Warrior
Posts: 1648
Joined: Thu Nov 02, 2000 10:00 am
Location: West Palm Beach, FL USA

Post by Warrior »

Originally posted by Don:
Ah, good old Warrior! We worked together so much on the MC that sometimes while positioning extra Matida's for a flank attack I'd be thinking of Steve's reaction! And wishing I could be a "fly on the wall" when they came in! :)
I REMEMBER THOSE MATILDA'S! They made me gnash my teeth, and if I had any hair I would have torn it out! :D
Retreat is NOT an option.

Image
mr172
Posts: 13
Joined: Mon May 15, 2000 8:00 am
Location: Roma Italy

Post by mr172 »

Ciao: come diventare un buon tester?
Well (in English for all the bunch). The first step is working like a proofreader. Double check all, the mistake could be hidden also in the best scenario and sometimes the designer himself could'nt know that the bug is here. Hidden building, stream under paths, units without ammo evrything.
Then play against not with the scenario. Search for weakness. There is an impassable route to get the flag? Try it, may be it was impassable only in the whisful thinking of designers. As tester you have two duties, demonstrate that the game is well balanced and in the same time demonstrate that you could anyway outbalance the game. Testers not designers are the worst friends of gamers.

Poi se vuoi ne parliamo più a lungo in privato. indirizzo mr172@hotmail.com

Massimo
User avatar
RockinHarry
Posts: 2344
Joined: Thu Jan 18, 2001 10:00 am
Location: Germany
Contact:

Post by RockinHarry »

..and finally don´t forget to send the "save Score" file. :D ...at least for me there´s valuable information to be found.

__________
Harry
RockinHarry in the web:

https://www.facebook.com/harry.zann
Wild Bill
Posts: 6428
Joined: Fri Apr 07, 2000 4:00 pm
Location: Smyrna, Ga, 30080

Post by Wild Bill »

Well, my turn.

Resisti, you have heard from the best of the best. The testers/designers who have responded to you here are the top of the line in my opinion.

Nearly all of them have tested for me and any success I enjoy in my efforts I attribute to this group of "unsung heroes."

Testing can be tedious. It can be frustrating. That is because you are dealing with an unfinished product.

But it can be also a lot of fun...a first peek at what is coming.

It also gives a real sense of satisfaction in knowing you have contributed to the gaming community, often in an anonymous way.

That is why I say that testers are selfless heroes. If the light ever shines on me, I think it should shine even brighter on these great gamers who give time and effort to make your wargaming experience so much fun!

I salute you all!

Wild Bill
Image
In Arduis Fidelis
Wild Bill Wilder
Independent Game Consultant
Wild Bill
Posts: 6428
Joined: Fri Apr 07, 2000 4:00 pm
Location: Smyrna, Ga, 30080

Post by Wild Bill »

Having said all that, let me pass to what I expect from those who work with me in testing scenarios.

1. I need durability. Too many volunteer...then disappear! :(
Count the cost before you begin.

2. I need communication. There are testers who are listening but never responding. I can't possibly know what is going on unless you write me.

This is perhaps my second biggest peeve: testers who don't communicate :mad:

3. I need thoroughness. Check it all. Testing is much more than just playing the battle.

It requires a sharp eye, a look at detail and the mechanics of the battle/campaign.

As has been stated, you start with the text. Read it. Check it first for grammar mistakes. note them.

Read it again to get the feel of the battle. Does it give you that? Is there too much? Is there too little? Does it sound accurate?

Now open the scenario. You've read the text. Now play the battle. Play it through. Note your score. Note anything wrong you see as you play. Make notes. If not, you'll forget.

I know this is different from what most do, but this is the best way.

Now that you have played it and gotten a your first impressions of it, open it in the scenario editor.

The first step is a hard look at the map. Any roads incomplete? Are there hidden building hexes?

Does it have the look and feel of real terrain? Does it fit the situation?

Next, look at the smaller operational map to the right? Little dots indicate troop locations. Do you see any that are still along the edge of the map? Should they be there, or did the designer miss them when deploying units?

After playing the scenario and looking it over, and IF you have the time, play it again.

Make more notes on what you find. Keep your final score.

Next, write out your report. Use common courtesy. DON'T say, "This is a piece of crap!" (even thought it might be). Be diplomatic.

Find something nice to say about the scenario before you begin, as we say here, "cracking" on the work.

Let your criticism be constructive.

Offer your own ideas on how it might be better.

Examples: It should be longer - shorter. Units should be closer or further away from the battle area. Reinforcements should come in earlier or later. Place reinforcement flags further away or closer.

Modify artillery or air support. If it is unbalanced, how could the designer make it more balanced (challenging, yet winnable)?

Remember, even if you make suggestions, the designer may not use them. Don't get offended. He has that right. His name is going on it.

If, however, you feel you are wasting your time trying to help a designer, just don't do anymore work for him.

These are a few haphazard thoughts for you Resisti to get you lined up for what you are about to do.

Good luck with it.

Wild Bill
Image
In Arduis Fidelis
Wild Bill Wilder
Independent Game Consultant
User avatar
Don Doom
Posts: 1984
Joined: Sat Sep 23, 2000 8:00 am
Location: Lost somewhere in the upper backwoods of Michigan!

Post by Don Doom »

Amen. ;)
Doom
Vet of the Russian General Winter
For death is only the begining
Don
Posts: 662
Joined: Wed Jul 12, 2000 8:00 am
Location: Elk Grove, CA (near Sacramento)

Post by Don »

One more thing - testing is one of the best ways to learn scenario design. I realized that testing for WB would teach me alot, so tested a few campaigns and it really paid off in seeing how he did things.

Tester's reports are also different, I think, depending on whether or not the tester is also a designer. I think it's natural that Alex would suggest improvements while Steve wouldn't, because Alex is also a designer and probably looks at the scenario a little differently. I don't see anything wrong with suggestions, myself. the designer can always not implement them.

Some testers also have a problem telling the designer if they don't like something - I think this is your job! If I think the map is crappy I say so - if I'm asked to test I figure it's my job to comment. But WB is right, it has to be done with something called tact, which I have yet to locate any of. Hopefully when I do I'll get some! :D
Don "Sapper" Llewellyn
mr172
Posts: 13
Joined: Mon May 15, 2000 8:00 am
Location: Roma Italy

Post by mr172 »

Just a nuance over the Wild Bill statement. I never read the text before playing. I will play with "ingenuity" because text often give me an other side of the hill look into designer ideas. So I first open the battle screen to the minimum zoom. General overlook of the battlefield, where I'm where I must go, then I zoom for the "trouble", trees along a road, buildings that can be useful for ambushes, dominanting hills...
Then I stop and start to think. what I'll do if I was the designer, where I put units, where and when the reinforcement could arrive... Then again what I, by sure, would'nt. This is usually what the designer has really done :D
Okay now check for the mistake. Scroll the map along the possibile way of approach, use a fast delay time and put the hex grid on the map, read carefully the black boxes. The full control with the editor could be done, as Bill says, after the battle.
Ciao
Massimo Rocca
Post Reply

Return to “Steel Panthers World At War & Mega Campaigns”