Search arc statistical test

[FatR]

I recently started abandoning search arcs as an experiment, using small 9-plane units flying ASW in coastal areas where enemy subs made their presence known. I haven't seen any decrease in detections so far and am gradually adopting "no search arcs" policy for all units.

[ade670]

Guys,

3 questions if I may:

1. IF I am on a coastal hex with 50% of the search arc on land and 50% search arc at sea - eg Townsville, will the AI automatically dismiss the 50% land element when plotting naval or asw search arcs.
I have always presumed that the routine would still plot both search arcs over the land mass hence the requirement to use the manual search arcs.

2. In the case of small sqdns 4 or less - surely setting up search arcs in a specific direction has got to improve the chances of spotting by if nothing else removing the odds of a the planes flying in the opposite direction to that which is required.

3. Finally, if a report of possible sighting is mention in the Ops report, by a search sqn on 'auto search', will the AI search script automatically fly the same radials on the next turn in order to improve the detection?

Any help with this increasingly grey area would be helpful.

Ade

[Sredni]

Well I guess this simplifies naval search setup heh. I wonder if it also effects asw.

Though I think we'll still want to setup search arcs for areas where we want to keep our search planes from overflying heavily defended bases. (I have a search plane hub near singapore in singakawan??. one group searching a zone right up to singapore to the northwest and another group flying farther north into the sea east of singapore. singapore has a heavy fighter defense I don't want to overfly)

I'm kinda disappointed to be honest. I like micromanaging search arcs. I like to press the Z key and see all the pretty search zones show up heh.

[crsutton]

ORIGINAL: Sredni

Well I guess this simplifies naval search setup heh. I wonder if it also effects asw.

Though I think we'll still want to setup search arcs for areas where we want to keep our search planes from overflying heavily defended bases. (I have a search plane hub near singapore in singakawan??. one group searching a zone right up to singapore to the northwest and another group flying farther north into the sea east of singapore. singapore has a heavy fighter defense I don't want to overfly)

I'm kinda disappointed to be honest. I like micromanaging search arcs. I like to press the Z key and see all the pretty search zones show up heh.

I am using no arcs for ASW and it seems to be doing just fine. Skill level and the range set for the ASW force seems to be more of a factor. The longer the range the more diluted the effort.

[Canoerebel]

After seeing plausible comments from many other players to this effect months ago, I discontinued setting search arcs in my game with Miller. At that point, we were well into 1944 in our game. For my game with Q-Ball, I haven't set an arc of any kind - search or ASW. As best I can tell my results are absolutely fine. Conclusion: I'm very glad to do away with the micromanaging hassle of setting arcs.

[castor troy]

ORIGINAL: Canoerebel

After seeing plausible comments from many other players to this effect months ago, I discontinued setting search arcs in my game with Miller. At that point, we were well into 1944 in our game. For my game with Q-Ball, I haven't set an arc of any kind - search or ASW. As best I can tell my results are absolutely fine. Conclusion: I'm very glad to do away with the micromanaging hassle of setting arcs.

I found the micromanagement great, with the assumption that it would be of course a huge benefit to have 12 Catalinas searching only 90 degrees instead of 12 Catalinas searching 360 degrees. Problem was, the enemy bombarded Akyab probably a dozen times without being spotted once while having dozens of patrols focussed exactly where the enemy was going in and out all the time. That made me more than sceptical and that was when I threw the archs overboard, it worked just fine with me ever since.

To be honest, I was amazed about the test even getting that many spottings WITH archs. I´m not surprised it got more without archs, even though it should achieve far more spottings with archs than without.

[witpqs]

ORIGINAL: ade670

1. IF I am on a coastal hex with 50% of the search arc on land and 50% search arc at sea - eg Townsville, will the AI automatically dismiss the 50% land element when plotting naval or asw search arcs.
I have always presumed that the routine would still plot both search arcs over the land mass hence the requirement to use the manual search arcs.

No, it will search the whole thing.

[Shark7]

ORIGINAL: Cribtop

Given the huge player time investment, setting search arcs should provide a meaningful benefit.

Well you do get some benefit, you don't waste time searchin over-land...but from these results that is it. [8|]

[CapAndGown]

I will just add to the chorus that I too have given up on search arcs. I was becoming rather skeptical already when I read Castor Troy's comments a while back and decided to give it a whirl. Sure enough, my results have been just fine if not better without the arcs.

This would be fine with me, except as one poster noted, there are times when you would like your planes to NOT search some area where the CAP is very heavy. Yet, I am now worried that using search arcs in this way will actually degrade my ability to spot ships.

[Lomri]

The part that is most striking about this test is the "wrong direction" conclusions compared to "no search arch". I tried googling to find the thread (with no luck), but I recall a developer saying that there was no negative side to having a search arch defined. You'd get better results in your search arch but no negative results outside of. This test, while a smallish test, still shows a huge negative to spotting outside an arch.

IF there was no downside and only upside to setting archs, I'd still use them. But if the gut feeling of folks and the limited testing shows that search archs are on par or worse, then search arches are a net-negative. Too bad too, I do get a kick out of the Z key showing my search coverage as I try to plug gaps.

[Smeulders]

I'm a fan of the search arcs as well, so I'd really like for them to have a noticeable impact on my searches. I'd be surprised if anyone ever said that there were no downsides to search arcs though. I'd think it only logical that if you tell your guys explicitly to search only a certain sector, they won't be in any others.  As for the sightings when search arcs were completely the opposite direction, maybe it has something to do with the targets being carriers. Remember, carriers increase their own detection when launching planes, so maybe they aren't the best TFs to use for this test.

[rader]

Ok, ran the test again at range = 10 hexes. I'm almost reluctant to post this, because these results are strange indeed.

Because someone mentioned CVs have a higher detection level, I used 4 groups of US ships this time. Each groups was composed of 2CA, 1DD, except one was 1CA, 1CL, 1DD. I put all the groups at a range of 10 hexes south of Rabaul like before.

Same group of Nells searching, 30% search/70% rest; 6000ft, searching at range 10. I used 9 samples per case this time.

Results (# TFs spotted)
Case 1. No search arc set (4/4/3/2/1/2/3/2/3) Mean = 2.67
Case 2. "Optimal" search arc (1/1/0/1/0/0/0/1/1) Mean = 0.56
Case 3. "Opposite/worst" search arc (2/1/1/0/1/2/0/1) Mean = 0.89

Clearely in this case, having no search arcs set (0-360) yielded significantly better results than either the optimal or worst search arc cases (p < 0.05).

However, for reasons I can't explain, setting the search arcs to the opposite direction actually improved the searches compared with the optimal search arc case [X(] (although this was not a statistically significant effect, p = 0.15).

What the heck is going on here? [&:]

My only guess is that there was another confounding factor.... maybe I got really bad weather for a large number of the "optimal" search cases? Unfortunately, I did not track this. Maybe there is a strange range effect? Perhaps in the last test, I was getting something strange in the tests because of the CV detection levels? I have no idea; I'm flabergasted. But I'm starting to think setting search arcs is a really bad idea! [8|]

I would appreciate if someone else could run these tests to help shed ligh on this and coroborate the results.

[rader]

ORIGINAL: Mistmatz

I like your test setup, but IMHO the sample size is by far too small too be statistically significant.

That comes out of the t-test. The differences were statistically significant (95% confidence interval) with the given sample size (when p < 0.05). But you're correct that for an accurate model, I would need to include the effects of weather and other differences between the cases. These cases weren't really the same, which does have some bearing on the significance.

[CapAndGown]

ORIGINAL: rader

But I'm starting to think setting search arcs is a really bad idea! [8|]

I've been thinking that for a while.

[Cad908]

Couple of suggestions:

1. Could you run with advanced weather effects off? I believe this might neutralize weather, did not see if you did this from previous posts.

2. Are you exiting the program completely after each test? The random number generator could be seeding each turn with the same starting point, thus not a true test.

Other than that, you seem to be confirming what many players are seeing in their games.

Thank you for your efforts.

[Cad908]

Duplicate deleted

[castor troy]

ORIGINAL: rader

Ok, ran the test again at range = 10 hexes. I'm almost reluctant to post this, because these results are strange indeed.

Because someone mentioned CVs have a higher detection level, I used 4 groups of US ships this time. Each groups was composed of 2CA, 1DD, except one was 1CA, 1CL, 1DD. I put all the groups at a range of 10 hexes south of Rabaul like before.

Same group of Nells searching, 30% search/70% rest; 6000ft, searching at range 10. I used 9 samples per case this time.

Results (# TFs spotted)
Case 1. No search arc set (4/4/3/2/1/2/3/2/3) Mean = 2.67
Case 2. "Optimal" search arc (1/1/0/1/0/0/0/1/1) Mean = 0.56
Case 3. "Opposite/worst" search arc (2/1/1/0/1/2/0/1) Mean = 0.89

Clearely in this case, having no search arcs set (0-360) yielded significantly better results than either the optimal or worst search arc cases (p < 0.05).

However, for reasons I can't explain, setting the search arcs to the opposite direction actually improved the searches compared with the optimal search arc case [X(] (although this was not a statistically significant effect, p = 0.15).

What the heck is going on here? [&:]

My only guess is that there was another confounding factor.... maybe I got really bad weather for a large number of the "optimal" search cases? Unfortunately, I did not track this. Maybe there is a strange range effect? Perhaps in the last test, I was getting something strange in the tests because of the CV detection levels? I have no idea; I'm flabergasted. But I'm starting to think setting search arcs is a really bad idea! [8|]

I would appreciate if someone else could run these tests to help shed ligh on this and coroborate the results.

this example is more in line in what happened in my game too. While your first test actually surprised me about NO archs only being slightly better than focussing all your ac into a small vector, your second test is more in line in what I would have said about my stomach feeling (is this even an English term?).

As it stands now I have done far better without archs as I can´t complain about spotting something at all... without archs...

[witpqs]

ORIGINAL: castor troy

this example is more in line in what happened in my game too. While your first test actually surprised me about NO archs only being slightly better than focussing all your ac into a small vector, your second test is more in line in what I would have said about my stomach feeling (is this even an English term?).

Pretty close Castor: "gut feeling"

[castor troy]

ORIGINAL: witpqs

ORIGINAL: castor troy

this example is more in line in what happened in my game too. While your first test actually surprised me about NO archs only being slightly better than focussing all your ac into a small vector, your second test is more in line in what I would have said about my stomach feeling (is this even an English term?).

Pretty close Castor: "gut feeling"

thanks [:)]

[Apollo11]

Hi all,

BTW, how did you do your testings?

Did you use original scenario (without changing anything) or you created your own scenario based on "Coral Sea" scenario?

How did you run your test (i.e. did you load WitP-AE and then load the scenario and tested, closed the scenario, loaded the scenario and tested again or you closed the WitP-AE as well)?

When I tested in UV and WitP the rule was always to close the program (i.e. UV, WitP) in between every single test!!!


Leo "Apollo11"