Search arc statistical test

[Sardaukar]

Might be worth doing couple more tests and then post on Tech Support, since it seems that search arcs are working exactly opposite as intended, which would suggest it's a bug. 

[tigercub]

i have been wasting a lot of time on those ACKs..at least we know about it now....please post to Tech Support to get the ball rolling...thanks guys for bringing this to light of Day.

Tigercub!

[d0mbo]

If the search arcs aren't working as intended, is it safe to say the same applies to the ASW arcs?

If so, i have not only wasted time putting my emily's and netties on search arcs, but MANY LB/FP groups on ASW duty as well!

Hope a Dev can alleviate my (our?) fears!

[Sardaukar]

I was wondering why AI was mauling my subs with planes..and I was not hitting much with my ASW...this might be the reason...[:(]

[olperfessor]

If setting the search arc is pointless, I can stop wishing that we could set the arc relative to a ship's direction (I wanted my TFs to be able to do ASW searches ahead of them, even when their courses changed after reaching waypoints.

Then again, I recall reading that search aircraft conduct searches within four hexes of their bases regardless of where (or whether) search arcs are specified. Not sure if this is the case.

[Sardaukar]

ORIGINAL: olperfessor


Then again, I recall reading that search aircraft conduct searches within four hexes of their bases regardless of where (or whether) search arcs are specified. Not sure if this is the case.

In addition to 4 hexes, they actually do search further too. Just that probability of spotting should be more probable withing arc and less outside. Unfortunately it doesn't seem to be the case if these tests are true.

In other words, if lucky and system worked as designed, there would be a chance of spotting outside the arc depending on range just as inside arc, but it'd be considerable lower.

[Chickenboy]

rader, et. al.,

Very interesting tests and discussion. I'm following with interest. Thanks for doing this...

[Walloc]

Thx for doing the tests rader.

If u per Sardurkar advice do more tests. It might interrestíng if it could be done in an aircraft with radar vs one with out.
PBY-5 vs PBY-5A comes to mind, as apparently ppl been getting better results with non radar equiped planes, then those with radar. Contrary to what one should think.

Kind regards,

Rasmus

[TR Shrum]

I have been concerned about setting Archs for Search and ASW and the Results of same for some time now. After reading these posts I went and check several of my In Progress games against the AI. I have been confused and frustrated by the poor results recieved by my AC when setting Archs and how much better the AI's results seem to be. I assumed it had something to do with the fact the Game Engine has to know something about where my units are in order to make things work. So I loaded several Saved Games in Two Player Mode and to my surprise I found that the AI never sets Archs. In every case (ASW or Search) no Archs are set, Max Range is set, and the % of AC assigned is either 33% or 66%; the balance at Rest. I've now ran several Turns of my most recent game without setting Archs of any kind for my uints and BINGO, I'm now getting much better Search and ASW contacts.

[morganbj]

ORIGINAL: Mistmatz

I like your test setup, but IMHO the sample size is by far too small too be statistically significant.

Well, larger samples would be nice, but the t test accounts for sample size through the use of degrees of freedom. It calculates the actual probabilities based on sample size. If the calculate probability (p) is below some value, usually .05, then it is statistically significant. So, I have to disagree with you there, Mistmatz.

But, this p is true only for the conditions set for the test. I think to draw the kind of conclusions you want, the test(s) would have to be set up with a little more control. E.g., I think each TF would have to be the exact same makup of ships. Several tests would then have to be conducted with varying detection distances, pilot experience and skill levels, and so forth. Then the same tests run with different TF makeup. Please understand, Rader, that what you did is not wrong, it's actually guite good. It's just not the complete picture - yet.

The thing to remember is that statistical significance does not mean practical significance. What does a .14 increase in probability really mean? One additional TF will get discovered 14 times every 100 searches. On average. And with TF makeup as it appeared in the test. And with the exact same range, with the same pilots, etc. That is why a little more testting should be done. And perhaps, Mistmatz, this is really what you were referring to, so I do not mean to discount your concern with my comments above.

This test did get significant results that should not be ignored as evidence that something may be a tad odd.

Nice job.


BTW, I teach statistics at the doctoral level at a university and really appreciate someone who uses these techniques for simple things like this. They don't always need to be used for some overblown, academic, pompus, research-about-nothing-important dissertation. I'm quite impressed, Rader. Drive on.

[Buck Beach]

ORIGINAL: Chickenboy

rader, et. al.,

Very interesting tests and discussion. I'm following with interest. Thanks for doing this...

I too am following the issue closely.

The search arcs, ASW searchs and pilot trainings are all very cool features and any problems need be addressed an fixed.

[SteveD64]

I've been setting search arcs and now that I know that they're not as effective I'm actually relieved that I won't have to micromanage them anymore.  Heresy I know!  Maybe when this is fixed there can be a pregame option to have effective search arcs or keep it the way it is now.

[hunchback77]

Rader, can you run the test with the Nells searching at 100% and no rest at 2000ft, same arc. I have a feeling that 30% Search/ 70% Rest may be causing the problems or it could be the 6000ft altitude setting. Thank you.

[witpqs]

ORIGINAL: Walloc

Thx for doing the tests rader.

If u per Sardurkar advice do more tests. It might interrestíng if it could be done in an aircraft with radar vs one with out.
PBY-5 vs PBY-5A comes to mind, as apparently ppl been getting better results with non radar equiped planes, then those with radar. Contrary to what one should think.

Kind regards,

Rasmus

IIRC the only difference there is wheels - strictly flying boat versus amphibious. Try the PBY-4 versus the PBY-5.

[witpqs]

Let's give them a bit to investigate this now that some tests have been run.

[Q-Ball]

Great work on this. I am very interested in the results.

I would love to see similar work on ASW, which is even more opaque, and difficult to replicate. I suppose that would take a long time, since hits can be random, but it would be great to finally know how to maximize ASW.

[Walloc]

ORIGINAL: witpqs

ORIGINAL: Walloc

Thx for doing the tests rader.

If u per Sardurkar advice do more tests. It might interrestíng if it could be done in an aircraft with radar vs one with out.
PBY-5 vs PBY-5A comes to mind, as apparently ppl been getting better results with non radar equiped planes, then those with radar. Contrary to what one should think.

Kind regards,

Rasmus

IIRC the only difference there is wheels - strictly flying boat versus amphibious. Try the PBY-4 versus the PBY-5.

Yes, my bad PBY-4 it is.

[ade670]

Guys

I'm a little frustrated with the lack of definitive answers from matrix on this .

Let's just keep things simple - forget altitude forget these rather spurious settings - search arcs dont work as intended or as described in the game manual - that = broken - please fix

Whilst I enjoy this game I'm increasingly of the opinion it is far from polished
On the information in the manual I have acted to set search arcs in my games - I want my valuable hours back !!!!

Thanks for the hard work so far highlighting this - the devs should send u a cheque

[CapAndGown]

ORIGINAL: ade670

Guys

I'm a little frustrated with the lack of definitive answers from matrix on this .

Let's just keep things simple - forget altitude forget these rather spurious settings - search arcs dont work as intended or as described in the game manual - that = broken - please fix

Whilst I enjoy this game I'm increasingly of the opinion it is far from polished
On the information in the manual I have acted to set search arcs in my games - I want my valuable hours back !!!!

Thanks for the hard work so far highlighting this - the devs should send u a cheque

I think this is uncalled for. The devs are doing a very good job. This is an extremely complex game. Of course there will be problems here and there. This problem is hardly a game killer. In fact, there are no game killing bugs I know of. (I would like to get the radar thingy fixed fast, though. [;)]) So chill out. With their limited resources they have to prioritize, but I am sure that this problem will be addressed in a reasonable amount of time.

[stuman]

ORIGINAL: ade670

Guys

I'm a little frustrated with the lack of definitive answers from matrix on this .

Let's just keep things simple - forget altitude forget these rather spurious settings - search arcs dont work as intended or as described in the game manual - that = broken - please fix

Whilst I enjoy this game I'm increasingly of the opinion it is far from polished
On the information in the manual I have acted to set search arcs in my games - I want my valuable hours back !!!!

Thanks for the hard work so far highlighting this - the devs should send u a cheque

We need to be patient. Just because we have raised an issue in the last 2 days does not mean that the Devs have had enough time to see this thread, drop everything they are doing in life, run all tests, etc and give us an answer.

And good on you Rader for doing some research. One of the many reasons this is one of the best games out there, and one of the best forum groups I have ever encountered.