Page 3 of 8
RE: Search arc statistical test
Posted: Fri Sep 17, 2010 5:35 am
by Sardaukar
Might be worth doing couple more tests and then post on Tech Support, since it seems that search arcs are working exactly opposite as intended, which would suggest it's a bug.
RE: Search arc statistical test
Posted: Fri Sep 17, 2010 7:35 am
by tigercub
i have been wasting a lot of time on those ACKs..at least we know about it now....please post to Tech Support to get the ball rolling...thanks guys for bringing this to light of Day.
Tigercub!
RE: Search arc statistical test
Posted: Fri Sep 17, 2010 8:04 am
by d0mbo
If the search arcs aren't working as intended, is it safe to say the same applies to the ASW arcs?
If so, i have not only wasted time putting my emily's and netties on search arcs, but MANY LB/FP groups on ASW duty as well!
Hope a Dev can alleviate my (our?) fears!
RE: Search arc statistical test
Posted: Fri Sep 17, 2010 8:25 am
by Sardaukar
I was wondering why AI was mauling my subs with planes..and I was not hitting much with my ASW...this might be the reason...[:(]
Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 9:40 am
by olperfessor
If setting the search arc is pointless, I can stop wishing that we could set the arc relative to a ship's direction (I wanted my TFs to be able to do ASW searches ahead of them, even when their courses changed after reaching waypoints.
Then again, I recall reading that search aircraft conduct searches within four hexes of their bases regardless of where (or whether) search arcs are specified. Not sure if this is the case.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 9:47 am
by Sardaukar
ORIGINAL: olperfessor
Then again, I recall reading that search aircraft conduct searches within four hexes of their bases regardless of where (or whether) search arcs are specified. Not sure if this is the case.
In addition to 4 hexes, they actually do search further too. Just that probability of spotting should be more probable withing arc and less outside. Unfortunately it doesn't seem to be the case if these tests are true.
In other words, if lucky and system worked as designed, there would be a chance of spotting outside the arc depending on range just as inside arc, but it'd be considerable lower.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 11:07 am
by Chickenboy
rader, et. al.,
Very interesting tests and discussion. I'm following with interest. Thanks for doing this...
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 11:30 am
by Walloc
Thx for doing the tests rader.
If u per Sardurkar advice do more tests. It might interrestíng if it could be done in an aircraft with radar vs one with out.
PBY-5 vs PBY-5A comes to mind, as apparently ppl been getting better results with non radar equiped planes, then those with radar. Contrary to what one should think.
Kind regards,
Rasmus
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 11:41 am
by TR Shrum
I have been concerned about setting Archs for Search and ASW and the Results of same for some time now. After reading these posts I went and check several of my In Progress games against the AI. I have been confused and frustrated by the poor results recieved by my AC when setting Archs and how much better the AI's results seem to be. I assumed it had something to do with the fact the Game Engine has to know something about where my units are in order to make things work. So I loaded several Saved Games in Two Player Mode and to my surprise I found that the AI never sets Archs. In every case (ASW or Search) no Archs are set, Max Range is set, and the % of AC assigned is either 33% or 66%; the balance at Rest. I've now ran several Turns of my most recent game without setting Archs of any kind for my uints and BINGO, I'm now getting much better Search and ASW contacts.
RE: Search arc statistical test
Posted: Fri Sep 17, 2010 12:56 pm
by morganbj
ORIGINAL: Mistmatz
I like your test setup, but IMHO the sample size is by far too small too be statistically significant.
Well, larger samples would be nice, but the
t test accounts for sample size through the use of degrees of freedom. It calculates the actual probabilities based on sample size. If the calculate probability (
p) is below some value, usually .05, then it
is statistically significant. So, I have to disagree with you there, Mistmatz.
But, this
p is true only for the conditions set for the test. I think to draw the kind of conclusions you want, the test(s) would have to be set up with a little more control. E.g., I think each TF would have to be the exact same makup of ships. Several tests would then have to be conducted with varying detection distances, pilot experience and skill levels, and so forth. Then the same tests run with different TF makeup. Please understand, Rader, that what you did is not wrong, it's actually guite good. It's just not the complete picture - yet.
The thing to remember is that statistical significance does not mean practical significance. What does a .14 increase in probability really mean? One additional TF will get discovered 14 times every 100 searches. On average. And with TF makeup as it appeared in the test. And with the exact same range, with the same pilots, etc. That is why a little more testting should be done. And perhaps, Mistmatz, this is really what you were referring to, so I do not mean to discount your concern with my comments above.
This test did get significant results that should not be ignored as evidence that something may be a tad odd.
Nice job.
BTW, I teach statistics at the doctoral level at a university and really appreciate someone who uses these techniques for simple things like this. They don't always need to be used for some overblown, academic, pompus, research-about-nothing-important dissertation. I'm quite impressed, Rader. Drive on.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 1:23 pm
by Buck Beach
ORIGINAL: Chickenboy
rader, et. al.,
Very interesting tests and discussion. I'm following with interest. Thanks for doing this...
I too am following the issue closely.
The search arcs, ASW searchs and pilot trainings are all very cool features and any problems need be addressed an fixed.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 3:45 pm
by SteveD64
I've been setting search arcs and now that I know that they're not as effective I'm actually relieved that I won't have to micromanage them anymore. Heresy I know! Maybe when this is fixed there can be a pregame option to have effective search arcs or keep it the way it is now.
RE: Search arc statistical test
Posted: Fri Sep 17, 2010 4:03 pm
by hunchback77
Rader, can you run the test with the Nells searching at 100% and no rest at 2000ft, same arc. I have a feeling that 30% Search/ 70% Rest may be causing the problems or it could be the 6000ft altitude setting. Thank you.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 4:41 pm
by witpqs
ORIGINAL: Walloc
Thx for doing the tests rader.
If u per Sardurkar advice do more tests. It might interrestíng if it could be done in an aircraft with radar vs one with out.
PBY-5 vs PBY-5A comes to mind, as apparently ppl been getting better results with non radar equiped planes, then those with radar. Contrary to what one should think.
Kind regards,
Rasmus
IIRC the only difference there is wheels - strictly flying boat versus amphibious. Try the PBY-4 versus the PBY-5.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 4:42 pm
by witpqs
Let's give them a bit to investigate this now that some tests have been run.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 5:06 pm
by Q-Ball
Great work on this. I am very interested in the results.
I would love to see similar work on ASW, which is even more opaque, and difficult to replicate. I suppose that would take a long time, since hits can be random, but it would be great to finally know how to maximize ASW.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 7:50 pm
by Walloc
ORIGINAL: witpqs
ORIGINAL: Walloc
Thx for doing the tests rader.
If u per Sardurkar advice do more tests. It might interrestíng if it could be done in an aircraft with radar vs one with out.
PBY-5 vs PBY-5A comes to mind, as apparently ppl been getting better results with non radar equiped planes, then those with radar. Contrary to what one should think.
Kind regards,
Rasmus
IIRC the only difference there is wheels - strictly flying boat versus amphibious. Try the PBY-4 versus the PBY-5.
Yes, my bad PBY-4 it is.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 8:57 pm
by ade670
Guys
I'm a little frustrated with the lack of definitive answers from matrix on this .
Let's just keep things simple - forget altitude forget these rather spurious settings - search arcs dont work as intended or as described in the game manual - that = broken - please fix
Whilst I enjoy this game I'm increasingly of the opinion it is far from polished
On the information in the manual I have acted to set search arcs in my games - I want my valuable hours back !!!!
Thanks for the hard work so far highlighting this - the devs should send u a cheque
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 10:24 pm
by CapAndGown
ORIGINAL: ade670
Guys
I'm a little frustrated with the lack of definitive answers from matrix on this .
Let's just keep things simple - forget altitude forget these rather spurious settings - search arcs dont work as intended or as described in the game manual - that = broken - please fix
Whilst I enjoy this game I'm increasingly of the opinion it is far from polished
On the information in the manual I have acted to set search arcs in my games - I want my valuable hours back !!!!
Thanks for the hard work so far highlighting this - the devs should send u a cheque
I think this is uncalled for. The devs are doing a very good job. This is an extremely complex game. Of course there will be problems here and there. This problem is hardly a game killer. In fact, there are no game killing bugs I know of. (I would like to get the radar thingy fixed fast, though. [;)]) So chill out. With their limited resources they have to prioritize, but I am sure that this problem will be addressed in a reasonable amount of time.
RE: Search by surface ships' floatplanes
Posted: Fri Sep 17, 2010 10:28 pm
by stuman
ORIGINAL: ade670
Guys
I'm a little frustrated with the lack of definitive answers from matrix on this .
Let's just keep things simple - forget altitude forget these rather spurious settings - search arcs dont work as intended or as described in the game manual - that = broken - please fix
Whilst I enjoy this game I'm increasingly of the opinion it is far from polished
On the information in the manual I have acted to set search arcs in my games - I want my valuable hours back !!!!
Thanks for the hard work so far highlighting this - the devs should send u a cheque
We need to be patient. Just because we have raised an issue in the last 2 days does not mean that the Devs have had enough time to see this thread, drop everything they are doing in life, run all tests, etc and give us an answer.
And good on you Rader for doing some research. One of the many reasons this is one of the best games out there, and one of the best forum groups I have ever encountered.