Lua/Mission settings possible effect on game speed?

[Exilemnord]

I have an odd problem. Have a pretty large game (Au count is 865 to start) that runs just fine for the first thirty or so minutes of game time. But thirty minutes in it just comes to a crashing slow pulse (2000 plus ms.) Then, when hostilities commence (this is set for a random moment over an 8 hour period,) everything speeds up to what it was at the beginning of the game DESPITE a heck of a lot more going on and never slows down again There is no particular scenario event being triggered at the slow down time that I can attach to its reason. Really, nothing much is going on at all.
So I suspect 1) There are certain missions I have set up that are slowly becoming a speed drag (being checked constantly) until they are triggered at hostilities (?) or LUA scripts I have written that are doing the same.
Possible? Anyone have any insight into anything like this?

[thewood1]

Might help if you posted the scenario. Otherwise its guess work. I know having a lot of missiles in the air can slow down scenarios. ABM activity especially can have a noticeable impact. PC spcs might also help normalize the discussion a little.

[Exilemnord]

Thanks Scenario attached

[thewood1]

I don't remember ever seeing that much lua in events as in this scenario. Especially looking at just a few of them, there are a lot of non-lua ways to do some of it. I think its a good guess that that much lua executing might have an issue. Not just because of the amount, but possibly because that much code needs some serious error-checking. I have an i9-13980 and it comes to a crawl when the events start hitting it. The Pulse time stays consistently above 300ms and spikes to 600ms. Anything above 100ms will start to make CMO a little sluggish.

[Exilemnord]

Thanks. Very much appreciated advice. Very good to know and yeah, lots of code. I'll look for efficiencies.
However, it doesn't explain why, when hostilities are declared an hour, two hours, three hours in...I'm suddenly back down to 20ms. It makes me think there's a specific piece of code or a mission that once triggered at hostilities commences vanishes and takes it's time delaying problem with it.

[thewood1]

I did notice that a lot of the triggers happen off of times and those timers are happening close to each other. It might be related to just too many timers at once.

One of the efficiencies you can look at is using the ops planner and dynamic missions. That will offload a bunch of code.

[Exilemnord]

oh! thanks! I wasn't familiar with it or perhaps it didn't exist when i began building this a couple years back. Having looked at it THAT will make about three quarters of this LUA code unnecessary (all the merchants transitioning through planned routes for instance.) Very helpful. Thank you.

[KnightHawk75]

" or LUA scripts I have written that are doing the same. "
---
None of your lua scripts do that (monitor anything), your triggers (monitoring what's in\or not a given area, or detecting) setup do though behinds the scenes though. The only problem I saw in your actual scripts is using global vars instead of local ones ('a = blabla' instead of 'local a = blabla'), and many of the scripts I eyeballed are acting upon objects without checking if they exist first (adding mags, helo's etc to things) which may or may not be ok given their execution context (which I didn't determinate for all when looking). The lua scripts involved are very simple, not recurring or recursive, and don't appear at a glace as the core of whatever is going on, but actions could be related to timing of when problems start happening.

I would recommend _looking at your logs_ when the slow down starts to see what if anything is going on, both activity wise in the game log, and any repeating errors that might be happening (be they from the game or the scripts) in the game error or script error logs.

For example.. on startup there are errors generated and on my system (about sides,weapons, and units not found that scripts are trying to act against).
The errors throw during on-scenario-load regardless:

Code: Select all

8/6/2023 3:55:34 PM -- B1307.19 -- Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index -- Scen : Puget Sound Environment -- DB :  -- 

Exception: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Stack Trace:    at System.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument, ExceptionResource resource)
   at DtrRowCollection.get_Item(Int32 i)
   at Command_Core.DAL.DBFunctions.GetWeapon(SQLiteConnection theConn, Weapon theWeapon, Int32 WeaponDBID, Scenario theScen, Boolean LoadComponents)
Call Stack & Error details: 
Error at 200274, Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index

lua log:
Function:ScenEdit_AddUnit (0)  Error:Side not found
Function:ScenEdit_AssignUnitToMission (0)  Error:UnitX was not found in the scenario unit list!
Function:ScenEdit_AddUnit (0)  Error:Side not found
Function:ScenEdit_AssignUnitToMission (0)  Error:UnitX was not found in the scenario unit list!
Function:ScenEdit_AddUnit (0)  Error:Side not found

Those may or may not be harmless overall

Then at around ~03:12-03:13Z (I pressed play and 15x and just let the game run) game basically halts with thousands of parallel-task errors getting logged (log went from a few dozen lines to 21meg in a few seconds):
The event that happens in game just prior to it locking up for me twice is:
"SH-60F Ocean Hawk - Determined as: Neutral (Classification by: Partridge Bank Lighted Buoy [Sensor: Mk1 Eyeball] at Estimated 2 nm)" in 2 of my attempts on a clean restart playing at 15x.
Followed by the following exceptions:

Code: Select all

8/6/2023 3:34:10 PM -- B1307.19 -- One or more errors occurred. -- Scen : Puget Sound Environment -- DB :  -- 

Exception: One or more errors occurred.
Stack Trace:    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IList`1 list, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, Action`1 body)
   at Command_Core.GameGeneral._Closure$__118-0._Lambda$__2(Side theSide)
Inner Exception: Object reference not set to an instance of an object.
Inner StackTrace:    at Command_Core.GameGeneral._Closure$__118-2._Lambda$__3(Contact theC)
   at System.Threading.Tasks.Parallel.<>c__DisplayClass31_0`2.<ForEachWorker>b__0(Int32 i)
   at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0`1.<ForWorker>b__1()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)
Call Stack & Error details: 
Error at 101293, 

But those didn't happen when playing at 5x. *shrugs*.

Other exceptions seen at points in time playing at 5x...(play 5x gets me past whatever goes on around 3:-12\13 at 15x).
They seem to occur every few seconds at do not appear to be lua generated but but behind the scenes exceptions.

Code: Select all

8/6/2023 4:10:17 PM -- B1307.19 -- Object reference not set to an instance of an object. -- Scen : Puget Sound Environment -- DB :  -- 
Exception: Object reference not set to an instance of an object.
Stack Trace:    at Command_Core.Weapon.IsNominallySuitableForThisTarget(Scenario theScen, Contact& theTarget, Nullable`1& TargetIsDestroyed)
Call Stack & Error details: 
Error at 200276, Object reference not set to an instance of an object.

Now as too the pluse time freakout after awhile (for me anywhere from 12min to 26min, but more often around 26min), when things goto crap pluse time wise for me. When this happens you will see (or I do see) the follow taken in the exception logs, there are like hundreds of duplicate entries per second when it happens and this is what is taking the extra pulse time (all the writing to the logs):

Code: Select all

8/6/2023 4:18:55 PM -- B1307.19 -- Index was outside the bounds of the array. -- Scen : Puget Sound Environment -- DB :  -- 

Exception: Index was outside the bounds of the array.
Stack Trace:    at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at System.Collections.ObjectModel.ObservableDictionary`2.Add(TKey key, TValue value)
   at Command_Core.ActiveUnit_AI.TargetThisContact(Contact theContact, Boolean AddedManually, Boolean PriorityTarget, _TargetingBehavior TargetingBehavior)
Call Stack & Error details: 
Error at 100041, 

Evironment: Runs were under build 1307.19, in editor mode, on an ancient srv2008r2 system,command.exe changed to normal priority while at main menu each time, before entering editor and loading and playing scene.

TLDR: Probably needs a dev to open this scene up and press play and monitor it at 5x or 15x till the ~30min mark to see what is actually causing the exceptions.

That said the scene rarely doesn't hang (and I mean stop progressing, actual exe is still responsive enough to exit) for me entirely at some point regardless of 15x or 5x, in fact in 6 attempts in the editor I only once got to 30min mark without a hard stop of time. Most of the time the log indicated the problems but, twice it just hung with no repeated errors in the log. Not sure any of this helps at all other then to say there are problems and often though not always you can see it in the logs while playing, and I don't think there is anything really wrong with your scripts - but they may be involved in the specific timing of when other underlying errors present themselves.

[Exilemnord]

wow man.. That is a deep dive and a lot of work thank you. Yes, simple lua though a good amount of it. I get no LUA error messages in the monitor (well, one, that I just can't be bothered to trace.) There are two events that seem to 'cause trouble. An Oceanhawk launching out of NAS Whidby, as you identified, and a cargo mission out of a southern California reserve base (Griffen 1.) If I disable both I get better performance longer. I have no idea why. Like none. I'm just going to rebuild those missions and related Events and see. Your note regarding triggers is worth thinking about. There are also possible issues around the changing of the Seawolf's magazines/weapons. I can't say why any of this would cause problems, but as you say, I will monitor the log files.
I really appreciate the effort you put it on this. My knowledge of OOL is fairly basic so my trouble shooting is limited.

[blu3s]

Lua errors appear in the LuaHistory file, under your CMOInstallationFolder/Logs/

[Exilemnord]

Thought I'd just finish this examination. The massive slowdown issue had, in the end, nothing to do with the LUA as it runs fine, even with the amount of it i have, after I fixed the issue. I deleted and rebuilt the two suspect air missions after running the scenario editor several times and noticing that (even with random ready times) as soon as the ASW mission "Down Jig" with the ocean hawks launched, the game came to a crawl (2000 ms pulse.). Got rid of it. Same thing happened with a cargo mission (Fast Umbrella.) Soon as those planes launched, same slowdown. I can not for the life of me tell why those two missions. I did notice an error saying that the units orders involved ignore plotted course when attacking and the mission said do not ignore and they were in conflict. But this seems a very minor problem. I had built both missions immediately after upgrading my database (?). Anyway - with hose two missions deleted, units deleted, mission rebuilt and units added back in - everything works fine.

[KnightHawk75]

Thought I'd just finish this examination. The massive slowdown issue had, in the end, nothing to do with the LUA as it runs fine, even with the amount of it i have, after I fixed the issue. I deleted and rebuilt the two suspect air missions after running the scenario editor several times and noticing that (even with random ready times) as soon as the ASW mission "Down Jig" with the ocean hawks launched, the game came to a crawl (2000 ms pulse.). Got rid of it. Same thing happened with a cargo mission (Fast Umbrella.) Soon as those planes launched, same slowdown. I can not for the life of me tell why those two missions. I did notice an error saying that the units orders involved ignore plotted course when attacking and the mission said do not ignore and they were in conflict. But this seems a very minor problem. I had built both missions immediately after upgrading my database (?). Anyway - with hose two missions deleted, units deleted, mission rebuilt and units added back in - everything works fine.

Glad to hear it, thx for follow up. Maybe, given you mentioned it was built over time (and various builds and db releases)... certain things were borked in how those missions\unit elements were saved at some point? ie at some point in the past maybe certain things\properties not getting written to saves, but along the way was fixed (for future new loading\savings etc) either with mission or units or their sensors etc etc, but maybe didn't make it into your otherwise 'working' save? idk *shugs*, but glad you were able to id some of the causes and a workaround with just rebuilding them.