DW2 Modding Roadmap

[elliotg]

Below is a summary of the current state of modding for Distant Worlds 2. It outlines what exists now and where we currently have gaps. We are eager to support modders and provide the tools and information they need. So we welcome your feedback and ideas, especially in the areas where we have gaps.

Distant Worlds 2 was written with modding in mind from the start. As much as possible, all the data is externalized in XML and text files. This approach helped in our development process, allowing separate teams to work on new content without needing lots of help from the development team (me).

So artists could integrate new models into the game, the story team could integrate new game events and the core game team could update the tech tree and components, all without needing development help.

As I see it there are 5 main areas of modding:
1. Data - configuration data and game text
2. Art Content - models, textures, sounds
3. Story and Events - linked events that unfold stories
4. Game Logic - code or data that strongly influences how the game plays
5. Mod Management - installing mods, handling multiple active mods

DW2 already handles items 1, 2 and 3 using various tools. Item 4 is only weakly supported. Item 5 is completely missing. We will be working to improve our support for points 4 and 5 in the coming weeks and months.

These areas are described in more detail below.

Data
Nearly all of the game objects are externalized in XML files: star and planet types, resources, races, governments, research, ship components, shiphulls, planetary facilities, creatures, troops

These XML files can be edited in the following ways:
1. DW2 Data Editor tool can edit many of the above objects
2. We provide XSD schema files for all of the above objects, so you can create your own tools if you wish
3. you can simply hand-edit the XML in NotePad

All of the game text is externalized, mostly in a single file (GameText.txt), but some also in the above-mentioned data files.

Galactopedia: the in-game Galactopedia help files are text files that use a very simple markup language.

Data Tools and Documentation

DW2 Data Editor_08032022.zip

(692.62 KiB) Downloaded 1476 times

DW2 XSDs.zip

(30.44 KiB) Downloaded 565 times

Distant Worlds 2 - Component Definitions - March 2022.pdf

(580.74 KiB) Downloaded 1590 times

Distant Worlds 2 - Planetary Facility Definitions.pdf

(392.34 KiB) Downloaded 475 times

Art Content
This includes the following items: ship models and textures, static images, UI icons, particle effects, sound effects.

DW2 uses Stride as it's rendering engine. Stride has a GameStudio tool that allows compiling art assets into bundle files that are included in the game. These bundle files thus include all of the above art content. However note that we do not use the latest version of Stride. Instead we use Xenko version 3.1.0.1 (Xenko was the former name for Stride).

We also have a tool (ship-tuner) to help modellers generate the necessary data from their models to get their ships and bases into the game. This tool helps setup all the data for weapon bays, hangar bays, engine bays, etc. This data is stored in the ShipHulls.xml file.

Star and Planet Types
All of the stars and planets are procedurally rendered. Modders can plug into this to make their own new types of planets or stars. This involves adding new entries to an XML file (OrbTypes.xml).

Art Tools and Documentation

Distant Worlds 2 - Making Ship Models - version 1.54.pdf

(643.27 KiB) Downloaded 643 times

Distant Worlds 2 - Ship Model Import Process - March 2022.pdf

(517.03 KiB) Downloaded 386 times

Stride Game Studio: https://www.stride3d.net/download/
ship_tuner-2.60-all.jar: TODO: download elsewhere...


Story and Events
DW2 has a powerful mechanism for creating stories with linked events. These stories are flexible and will mold themselves to any new galaxy - they are not tied to a single savegame.

Story events are contained in an XML file (GameEvents.xml). These game events have logic to place them at the right location in the galaxy at the start of the game. They also have trigger conditions that control when they occur. And then they have multiple actions that can happen once they are triggered, including prompting the player for decisions.

We use this approach to do all of our internal story development, so there are lots of pre-existing examples of how to use this in the GameEvents.xml file. The game events are also comprehensively documented in "Distant Worlds 2 - Game Events - 5 March 2022".

None of these game events require writing any code. You simply edit entries in an XML file, using the provided Data Editor tool.

So while I understand that many modders may want to dive straight into changing game code, I would urge you to first take a good look at the tools already available that may do most of what you want. Game Events in particular cover a lot of ground that modders will find useful.

Story Tools and Documentation

Distant Worlds 2 - Game Events - 5 March 2022.pdf

(810.5 KiB) Downloaded 407 times

Game Logic
One mechanism that can already be used to strongly influence gameplay are Empire Policy files. These are XML files that contain settings for each playable race. They influence how the race plays the game: what they value and prioritize, thresholds for various decisions and other data. The policy files can be found in the data/policy subfolder. They are named by race, e.g. Human.xml. These files contain the default settings for each race.

The policy settings in these files correspond to the Empire Policy screen in the game. You can hover over each setting in the Policy screen to get an explanatory tooltip.

Connecting to Game Code
The code in DW2 is currently obfuscated. However we plan to deobfuscate most of this to allow access to code modding.

Mod Management
DW2 does not currently have a good approach to handling multiple mods at the same time. In DW Universe we had a separate folder for each mod, and you could select which mod you wanted to use - one at a time.

That's pretty much where we are at the moment. But we want to improve this a lot. We need a good way to revolve conflicts when mods use the same files, indicating which takes precedence, etc.

We are also missing a user-friendly mechanism for discovering and installing mods. Placing mods in their own subfolder is fine, but we need to wrap this with nice UI to manage things.

[BTAxis]

On the subject of management, I think it would be a big win if you could tie it into the Steam Workshop. Obviously you're going to want a mechanism aside from that to allow non-Steam players to manage mods, but I think a big chunk of your player base will be on Steam and the Workshop offers an excellent platform for modders and mod users alike.

For non-Steam modding, perhaps you could leverage existing solutions such as Mod Organizer 2 or the XCOM AML. Both programs are licensed under the GPL.

[btd64]

Good stuff Elliot. Needs to be stickied somewhere....GP

[Scott2933]

Would one of those categories include UI elements? I'd like to be able to mod the button images along the bottom of the Select panel, for example.

[Deathawaits4]

Hello Elliot,

Its me, that annoying guy that was harrasing you for code mods the last few months. Thanks for the headsup!

I also have a suggestion for modding compatibility espacially for XML.

Rimworld is one of the best examples on how it should be done. Basically there is something called "patch operation" and "xpathing"

You can create a new xml that has certain operations and requirements attached, that get read by the engine and modify xml files on runtime. (this is only a simple explanation, but the things that can be done with it are huge)

I would highly suggest checking out this post: https://rimworldwiki.com/wiki/Modding_T ... Operations

once you read into it, you will realize that its basically the holy grail of compatibility between many mods and has proven to be the foundation of rimworlds modding scene. I dont know of any other game that uses this system, atleast not in xml.

One example is: PatchOperationFindMod. With this you check for loaded mods and only if another mod is present, the actual operation runs. This can be combined with many others like: Patchoperationconditional, basically this will check for a xml node and if present it will atach itself to that xml node

"PatchOperationConditional can branch logic based on a match/nomatch. The following example patch adds a <comps> node if it does not yet exist, and adds itself"

This basically allows mod authors to create cross mod compatibility and also allows mod authors to attach their own xml snippets to other mod authors xml node. Depending on load order you could add snippets to snippets indefinetly.

The whole system is also fairly easy to implement code wise compared to some very complex api that is supposed to handle this and provides much deeper control for authors.

Hope i could be of some help!

[brucethemoose]

Hello Elliot,

Its me, that annoying guy that was harrasing you for code mods the last few months. Thanks for the headsup!

I also have a suggestion for modding compatibility espacially for XML.

Rimworld is one of the best examples on how it should be done. Basically there is something called "patch operation" and "xpathing"

You can create a new xml that has certain operations and requirements attached, that get read by the engine and modify xml files on runtime. (this is only a simple explanation, but the things that can be done with it are huge)

I would highly suggest checking out this post: https://rimworldwiki.com/wiki/Modding_T ... Operations

once you read into it, you will realize that its basically the holy grail of compatibility between many mods and has proven to be the foundation of rimworlds modding scene. I dont know of any other game that uses this system, atleast not in xml.

One example is: PatchOperationFindMod. With this you check for loaded mods and only if another mod is present, the actual operation runs. This can be combined with many others like: Patchoperationconditional, basically this will check for a xml node and if present it will atach itself to that xml node

"PatchOperationConditional can branch logic based on a match/nomatch. The following example patch adds a <comps> node if it does not yet exist, and adds itself"

This basically allows mod authors to create cross mod compatibility and also allows mod authors to attach their own xml snippets to other mod authors xml node. Depending on load order you could add snippets to snippets indefinetly.

The whole system is also fairly easy to implement code wise compared to some very complex api that is supposed to handle this and provides much deeper control for authors.

Hope i could be of some help!

This, so much this ^.

Rimworld patching lets multiple mods touch the same vanilla xml file gracefully. It lets mods patch each other, based on all sorts of conditions. And it still works when other mods, or the core game, change those targeted files. Its simple, elegant, and extremely powerful. To summarize, its nothing short of a godsend, and it would fit DW2 like a glove.

A bit of context: Rimworld had a dark age of modding during Alpha, before patching was implemented. Mods overwrote each other and overwrote vanilla files, so all sorts of stuff would break with even the littlest update. Even pure xml mods broke each other in all sorts of ways, which in addition to making the player experience rather poor, also created some cross-modder drama. And even after patching was implemented, it took modders a bit to "learn" xml patching, as they were used to simply overwriting files. Hence I would urge Matrix to implement xml patching sooner rather than later, otherwise DW2 modders will pick up bad habits.





Also, I made a quick VSCode config for those XSD files. Use with the Red Hat XML extension, and just paste it in settings.json:

Code: Select all

 "xml.fileAssociations": [
        
      {
        "pattern": "Artifacts.xml",
        "systemId": "ArtifactList.xsd"
      },
      {
        "pattern": "ColonyEventDefinitions.xml",
        "systemId": "ColonyEventDefinitionList.xsd"
      },
      {
        "pattern": "ComponentDefinitions.xml",
        "systemId": "ComponentDefinitionList.xsd"
      },
      {
        "pattern": "CreatureTypes.xml",
        "systemId": "CreatureTypeList.xsd"
      },
      {
        "pattern": "FleetTemplates.xml",
        "systemId": "FleetTemplateList.xsd"
      },
      {
        "pattern": "GameEvents.xml",
        "systemId": "GameEventList.xsd"
      },
      {
        "pattern": "Governments.xml",
        "systemId": "GovernmentList.xsd"
      },
      {
        "pattern": "OrbTypes.xml",
        "systemId": "OrbTypeList.xsd"
      },
      {
        "pattern": "PlanetaryFacilityDefinitions.xml",
        "systemId": "PlanetaryFacilityDefinitionList.xsd"
      },
      {
        "pattern": "Races.xml",
        "systemId": "RaceList.xsd"
      },
      {
        "pattern": "ResearchProjectDefinitions.xml",
        "systemId": "ResearchProjectDefinitionList.xsd"
      },
      {
        "pattern": "Resources.xml",
        "systemId": "RescourceList.xsd"
      },
      {
        "pattern": "ShipHulls.xml",
        "systemId": "ShipHullList.xsd"
      },
      {
        "pattern": "SpaceItemDefinitions.xml",
        "systemId": "SpaceItemDefinitionList.xsd"
      },
      {
        "pattern": "TroopDefinitions.xml",
        "systemId": "TroopDefinitionList.xsd"
      }
    ]

Oddly enough, its already finding some errors in untouched vanilla files. Here's one such example in ComponentDefinitions.xml:

Code: Select all

Invalid element name:
 - FiringEffectScaling

One of the following is expected:
 - FiringEffectFilepath
 - ArmorImpactEffectFilepath
 - HullImpactEffectFilepath
 - FiringSoundEffectFilepath
 - ArmorImpactSoundEffectFilepath
 - HullImpactSoundEffectFilepath
 - ArmorImpactEffectScaling
 - HullImpactEffectScaling
 - FiringSoundEffectVolume
 - ArmorImpactEffectTintColor

Error indicated by:
 {the schema}
with code:

The line is "<FiringEffectScaling>2.5</FiringEffectScaling>"

[Arkblade]

Congrats on the release! I am enjoying the more advanced game than DW1.

btw, Can we change the game font?
This is a real acute problem in some cultures.

Unless you guys plan to support so many languages, non-English languages will not have a font in this game.
This is a big problem for those who are trying to introduce the game in those languages.
If the game wasn't so complex, it might have been better to just use English, but it still has great complexity in DW2 (which of course is what makes it so much fun).

This is sad because the game text mechanism itself seems to be aware of i18n, including the CJK sphere.

[imperialWorldX]

Ooo i can finally create the Asimov Foundation Series Mod i always wanted

[Spidey]

Is it actually possible to unpack the asset archives or is the Stride bundling strictly one-way?

Have to say, so far I'm not seeing how this game is more designed for modding than DW1. Seems the opposite, to be frank. Maybe it's great for adding additional stuff but tweaking existing files does not feel like a trivial pursuit.

[eddieballgame]

Wondering if we will be allowed to create new galaxy types & adjusting the 'stars' numbers.
For example, a system with only 100 stars.

[LeFoureur]

@elliot : Thank you very much for taking into account the requests of the community, especially regarding the non-obfuscation of part of the source code.

The community will do great things for sure and it's thanks to you. (Erik & You)

I remember making this request early on and was pleasantly surprised that I didn't receive a direct refusal, but that I was able to bring some thought to this code release.

I think we're lucky to have you, really.

[Deathawaits4]

Wondering if we will be allowed to create new galaxy types & adjusting the 'stars' numbers.
For example, a system with only 100 stars.

Just wanna barge in, once code mods are available, the things we can do are near endless. E.g making a 100star galaxy will be one of the easier things (if it is not already exposed in xml)

[Osito]

First and foremost, I agree with the other posts that mods need to be compatible with each other. That lack of compatibility was a major problem with modding DW:U. I imagine this is also a priority for Matrix.

Second, I'd like to see a proper map creator. I made a Milky Way galaxy map for DW:U, and it was a really tedious exercise placing all the systems individually. Allowing the data to be drawn from an xml file would be much better. This also needs the ability to design the individual star systems (i.e. planets and the like). I don't like to mention the S-word, but Stellaris had this functionality in vanilla, although it was nowhere near perfect. And it hopefully goes without saying that the new starmaps needs to work with mods. I did quite a lot of work creating star maps in DW:U and Stellaris, and I can talk in detail about some of the issues, if you wish.

Third, it would be nice if mods could break some of the normal game rules. For example, in DW:U, certain planet types were uncolonisable. I'd like to see the ability to make them colonisable with mods, or at least to create new planet types. A similar comment applies to planetary facilities. If I recall correctly, In DW:U, you could only create new facilities that used the abilities of facilities that were already in the game. It would be interesting if facilities could be created that have different abilities from those already in the game. I appreciate that the coding may preclude something like this.

Finally, I would like to be able to create new star types.

Thanks.

[eddieballgame]

Wondering if we will be allowed to create new galaxy types & adjusting the 'stars' numbers.
For example, a system with only 100 stars.

Just wanna barge in, once code mods are available, the things we can do are near endless. E.g making a 100star galaxy will be one of the easier things (if it is not already exposed in xml)

No 'xml' files per editing number of stars, etc, that I can find & thank you for your response.

[LoneWolfGK]

Hi,

thank you for the modding help so far.

Is there any way to mod out nebulas and if so, how and where.

Also where to mod corruption, its too high at the moment.

Thank you.

[Jadey]

When is the exploration music folder triggered? I've figured out everything else besides exploration and the extra folder.

Most of is pretty clear cut:
Discovery: music that plays when you search ruins or whatnot, three levels of intensity.
Action: combat, three levels.
Industry: System View
Peaceful: Galaxy View
Research: Research Window
Ship Design: Ship designer

[Miletkir]

This includes the following items: ship models and textures, static images, UI icons, particle effects, sound effects.

DW2 uses Stride as it's rendering engine. Stride has a GameStudio tool that allows compiling art assets into bundle files that are included in the game. These bundle files thus include all of the above art content.

Which raises the question: How to do it? Or is it meant that the decompiling is a planned feature?

[LeFoureur]

This includes the following items: ship models and textures, static images, UI icons, particle effects, sound effects.

DW2 uses Stride as it's rendering engine. Stride has a GameStudio tool that allows compiling art assets into bundle files that are included in the game. These bundle files thus include all of the above art content.

Which raises the question: How to do it? Or is it meant that the decompiling is a planned feature?

Yes, there was a discussion on Matrixgames or Reddit (I don't really remember) where several people asked that there be no code obfuscation to be able to decompile and push the modding to infinity

Let's be clear, if this really happens (and I have no doubt that it will), Distant Worlds 2 will become a must-have and I hope that (relevant) community contributions can eventually be integrated into the game.

[Arcurus]

Thx for the guide, but why is the data editor not signed with some certificate?

[Arcurus]

I edited the ComponentDefinitions, but the changes showed only up after starting two times from new.
The changes don't show up in saved games.
Is it possible to change the components also for saved games? If so how?