You do not need to be an expert on game mechanics to make assumptions on whether the data you observe is self consistent and sufficient.
My impression is that you attempt to estimate the # of ships/AA mounts that enable you to build, how you call it, an AA TF and use it as trap(bait?) against naval strikes. To draw conclusions what number is sufficient for survival as well as effect, you seem to tie the number of AA mounts to the number of attacking aircraft, and from there attempt to deduce how many ships you need to be successful without generating hits on your ships.
So basically you observe following variables:
- # of AA guns/# of ships
- # of attacking a/c
and then establish a causal relation of those numbers to the number of hits on your ships (and maybe to the a/c shot down).
But what you ignore is that number of hits is the result of many more causal relationships. Just to name a few, theres many more):
- # of AA guns/# of ships
- # of attacking a/c
- TF DL/MDL
- a/c pilot skill
- initial a/c pilot fatigue
- strike distance (also affecting fatigue)
- squadron commander skill
- TF leader skill
- ship attributes (IIRC mvr&speed and maybe displacement)
- ship commander skill
- crew exp
- attacking a/c type
- attcking a/c loadout
- weather
If you are only observing part of those variables, any observation you make is worthless.
I am aware that one cannot know all variables impacting a specific combat situation except with access to the code (I would never claim I do), but even without that knowledge you can set a framework that minimizes the impact of variables you do not know or want to rule out as a factor. What you need to do is keep the test setup as static as possible, and only change those variables you want to observe. But depending on the variables you do not know, and depending on th ecomplexity of the situation you want to observe, you have to make several dozens of repetitions to get statistically significant data.
But even then, you only have a rough baseline for a very specific set of variables, so tests would not support you in predicting results reliably if the variables significantly diverge from your sandbox setup. And that such variation will happen in a campaign is pretty much a given.
What I DON'T KNOW is if 25 bombers attack one ship (as with the second combat report), do they perform their attack simultaneously or are their broken down into smaller packets, which wait for their turn for the attack?
In most instances they are broken down into smaller packets and attack sequentially. You can see the composition of the packets in the combat animations.
If it is the latter, and at one time only i.e. four dive-bombers attack CA Chokai, then the CA can concentrate all her AA fire on those four bombers provided the guns are not limited in their targeting by Right Side/Left Side placement.
No, because even in the above described sequential attack you will notice a target saturation effect. The more a/c attack, the less concentrated is the AA.
That leads us to a question what is the direction from which the dive-bombers attack a ship. Combat animation only gives you direction information when a torpedo bomber attacks. A dive-bomber has no direction, it always attacks a ship from overhead(?), thus it may face the AA from all the AA guns on the ship.
Please take this with a grain of salt as I have not seen the code. My personal assumption is all attacks face one or more sides of a ship (Front/Rear/Right Side/Left Side/All), depending on a dice roll. The details of this have never been revealed.
You raise good, valid, and interesting questions, but please be aware that even if answered they will not make you tests, as you currently seem to set them up, more conclusive.
. In reality most planes were shot down by 40/30/25/20/13mm guns than by dp big calibers. And as no propably in game only few hits (1-3) from 20mm+ were manytimes enought to shot down most planes, except 4e heavy bombers (for these were 30mm+ 
