[1278.1] Big scenario performance - any comparisions?

[nointernet]

Hi, I cannot compare to non-beta at the moment, but did any of you tried to play big / complex scenarios like Baltic-22 ( https://www.matrixgames.com/forums/view ... 7#p4999687 ) or Don of new Era ( DLC / Live ) using beta? I'm getting suboptimal experience during "the main escalation moment" - my CPU literally melts and CMO becomes unresponsive for longer time, often I get OOMs, some time I see all cores used. After restart it seems to be a little better, but still too slow for "good experience".

I've observed several that CMO is close to exhausting 32-bit addr space or has mem leak? too, I saw tweet about 64-bit one some time ago.

All game options for performance are enabled (to reduce drawed objects).

Is it a known issue? Or is it me expecting too much (I realize simulating "everything related to warfare" in such big-count scenario is computive-intensive)? I could gather some profiles if some dev points me to proper tool/format.

Intel i5 7600k (4x cores @ up to 4.5Ghz, Turbo Boost & overclocked automatically by motherboard ) + 32 GB RAM.

Some examples:

9/25/2022 8:11:41 PM -- B1276.1 -- Exception of type 'System.OutOfMemoryException' was thrown.
Exception: Exception of type 'System.OutOfMemoryException' was thrown.
Stack Trace: at Command_Core.GeoTIFF_Tile.ReadValue(Double theLon, Double theLat)
at Command_Core.LandCover.nvYDVi5UENgZ5YPPXAmy(Object , Double theLon, Double theLat)
at Command_Core.LandCover.GetLandCoverAtThisPoint(Double theLat, Double theLon)
at Command_Core.Sensor.yD8mbJ8IreIWhqdPsyaI(Double theLat, Double theLon)
at Command_Core.Sensor.MaxDetectionRangeOnThisTarget_Visual(ActiveUnit myParent, ActiveUnit theUnit)
Call Stack & Error details:
Error at 100719,


9/25/2022 8:11:41 PM -- B1276.1 -- Exception of type 'System.OutOfMemoryException' was thrown.
Exception: Exception of type 'System.OutOfMemoryException' was thrown.
Stack Trace: at Command_Core.GeoTIFF_Tile.ReadValue(Double theLon, Double theLat)
at Command_Core.LandCover.nvYDVi5UENgZ5YPPXAmy(Object , Double theLon, Double theLat)
at Command_Core.LandCover.GetLandCoverAtThisPoint(Double theLat, Double theLon)
at Command_Core.Sensor.MaxDetectionRangeOnThisTarget_IR(ActiveUnit myParent, ActiveUnit theUnit)
Call Stack & Error details:
Error at 100720,


9/25/2022 8:11:42 PM -- B1276.1 -- Exception of type 'System.OutOfMemoryException' was thrown.
Exception: Exception of type 'System.OutOfMemoryException' was thrown.
Stack Trace: at Command_Core.GeoTIFF_Tile.ReadValue(Double theLon, Double theLat)
at Command_Core.LandCover.nvYDVi5UENgZ5YPPXAmy(Object , Double theLon, Double theLat)
at Command_Core.LandCover.GetLandCoverAtThisPoint(Double theLat, Double theLon)
at Command_Core.Sensor.yD8mbJ8IreIWhqdPsyaI(Double theLat, Double theLon)
at Command_Core.Sensor.MaxDetectionRangeOnThisTarget_Visual(ActiveUnit myParent, ActiveUnit theUnit)
Call Stack & Error details:
Error at 100719,

Another one:

9/26/2022 3:17:36 PM -- B1277.1 -- Exception of type 'System.OutOfMemoryException' was thrown.
Exception: Exception of type 'System.OutOfMemoryException' was thrown.
Stack Trace: at Algorithms.PathFinderFast..ctor(Int32[,] grid)
at Command_Core.CostBasedPathfinder.SolvePF_Coarse(ActiveUnit theUnit, Double StartLat, Double StartLon, Double EndLat, Double EndLon, Single CurrentHeading, Single DegreeBuffer, Single ProximityThreshold_Deg, List`1& ProvidedPiers, Single& PercentComplete, Boolean IsMissionPlannerRequest, Boolean AllowFineGrainedNav)
Call Stack & Error details:
Error at 101333,


9/26/2022 3:17:46 PM -- B1277.1 -- Exception of type 'System.OutOfMemoryException' was thrown.
Exception: Exception of type 'System.OutOfMemoryException' was thrown.
Stack Trace: at Algorithms.PathFinderFast..ctor(Int32[,] grid)
at Command_Core.CostBasedPathfinder.SolvePF_Coarse(ActiveUnit theUnit, Double StartLat, Double StartLon, Double EndLat, Double EndLon, Single CurrentHeading, Single DegreeBuffer, Single ProximityThreshold_Deg, List`1& ProvidedPiers, Single& PercentComplete, Boolean IsMissionPlannerRequest, Boolean AllowFineGrainedNav)
Call Stack & Error details:
Error at 101333,


9/26/2022 3:21:11 PM -- B1277.1 -- Exception of type 'System.OutOfMemoryException' was thrown.
Exception: Exception of type 'System.OutOfMemoryException' was thrown.
Stack Trace: at NonBlocking.DictionaryImpl`3..ctor(Int32 capacity, DictionaryImpl`3 other)
at NonBlocking.DictionaryImplRef`3.CreateNew(Int32 capacity)
at NonBlocking.DictionaryImpl`3.ResizeImpl(Boolean isForReprobe)
at NonBlocking.DictionaryImpl`3.PutIfMatch(TKey key, TValue newVal, TValue& oldVal, ValueMatch match)
at Command_Core.SeaIceProvider.FsVYECNFrk5dDAtZuDht.stJNFScisZa(String , Boolean )
at Command_Core.SeaIceProvider.sugOB94ydpgvNviEXPIh(Object , Object , Boolean )
at Command_Core.SeaIceProvider.PointIsUnderIce(Double theLon, Double theLat)
Call Stack & Error details:
Error at 101100,

[nointernet]

My options are attached.
Example, when everything is ok (playable) the pulse time is ~60..180ms with AU count ~2100
when it is not pulse time goes into 600..1300ms with AU count ~2200

[thewood1]

"CPU literally melts"

Are you sure you mean literally...

Also, latest build is 1177.1. Just to avoid confusion.

[thewood1]

btw, thats a pretty old CPU. I think it was released 15 years ago. One thing I found that provides a little more push for large scenarios is fast RAM. That CPU is running on an older and slower bus. Overall, it probably just doesn't have processing power of a modern CPU. That would be my guess.

Related, scenario size is not unlimited. Huge scenarios might play a little better in the betas because of the new high compression setting, but when a lot of missiles are in the air, its probably going to be the same.

Also, turn off auto-save. In large scenarios, it can chew cycles.

[nointernet]

Hi I've made an error/typo related to CPU name, it's fairly recent i5 7600k @ 3.8Ghz (+18% auto-boost =~ 4.5Ghz)

[nointernet]

OK, I've made another typo - it's about v1277.1 of course.

Partial resolution of the slow performance:
1) main bottleneck: Map Settings -> Show range symbols for ... -> currently selected unit
2) disable sounds (both music and effects) - not sure if it helps
3) select only raw message - not sure if it helps

... and somehow it started to be playable through most intensive battles, but after some time I've got a CMO hang (another bug I think, I'm going to start another thread).

[thewood1]

I've played through twice now. It runs incredibly fast for the first few hours. Like 60:1 or faster through the first few hours. Once missiles start flying (hundreds of them), its slows to 2-3:1 time compression. Its not just the size of the scenario, its the large number of missiles, active sensors, and relatively dense units.

I have a i9-12900HX, 3080 ti, 32 GB RAM, and excellently cooled laptop. When missiles fly, I'm at over 50% CPU utilization on average across all cores with some maxed out. Use that as a reference.

[nointernet]

I've played through twice now. It runs incredibly fast for the first few hours. Like 60:1 or faster through the first few hours. Once missiles start flying (hundreds of them), its slows to 2-3:1 time compression. Its not just the size of the scenario, its the large number of missiles, active sensors, and relatively dense units.

I have a i9-12900HX, 3080 ti, 32 GB RAM, and excellently cooled laptop. When missiles fly, I'm at over 50% CPU utilization on average across all cores with some maxed out. Use that as a reference.

Thanks, that's good reference. Your CPU has 8 cores/16 threads, I was under impression that CMO is capable of using of up to 3 of them (and not more, I've never saw more than 3 used to be honest).

Are you really sure you have >50% CPU on all of the cores? (It might me push me into buying some 8c/16t too)

[thewood1]

Its actually 16/24 on CPUs, with on 8/16 being full performance. I'll double check the usage with xtu.

[thewood1]

As you can see with this screen cap, its using significant portions of all cores.

Screenshot 2022-09-28 045406.jpg (207.61 KiB) Viewed 541 times

You can see the initial spike of loading CMO hits only a few cores. Then an all-core spike when I load the scenario. When I hit start at double-flame, all cores chip in with a couple a little higher than the others. You then see the big change when all missiles are in the air. A few cores get as high as 60, but all cores are tracking between 40 and 50. Its using all performance cores and all efficiency cores. I've been saying this for years. For CMO, more cores are better.

[Dimitris]

CMO in general has much higher multi-core CPU utilization compared to CMANO (we mentioned this at CMO's original release), and the Tiny branch has further expanded this.

One of the byproducts of parallelized salvo evaluation, for example, is that now multiple potential shooters in a salvo will evaluate their DLZ checks (a CPU-expensive operation) each on his own personal thread. This naturally leads to both higher CPU utilization and a greater performance penalty when there are multiple potential targets being evaluated for interception.

I haven't observed any consistent OOM exceptions yet, though. Anyone else seen this?