Programming language and design of AI in DW2?

[zebanovich]

Hi, I would like to ask which programming languages are used for AI in DW2?

Is the AI framework written in one language and game specific AI logic in another?
Do you maintain reusable AI framework or is the AI designed specifically for DW2?

---

I believe maintaining a separate AI framework designed for 4X games is useful for good AI, which could then be reused and adjusted for any game in genre as needed, if you're of same opinion what do you think would be the most appropriate language for this?

If you don't think so, what is your argument?

Anything else you could add to AI design specific for 4X games or specific for DW2 is welcome.
Thank you for your time.

[Jorgen_CAB]

That depend what you define as AI as most AI in games are more or less scripted with weighted values for different situations and then some randomization thrown in for good measure. For the most part how do you suppose to take the same AI from different games and apply them when the games usually operate on many different scales and mechanics.

We are not talking about any form of learning AI or artificial AI or neural net AI. A neural net AI would probably take more resources than creating the game to get right and as soon as you change any game mechanic you have to more or less start from scratch or at least put some vast resources into relearning the AI. Until we have far better more potent computer I don't see such AI being economically viable.

I would like top understand what portion of an AI can be reused between two very different games?

[zebanovich]

Diplomacy is one example where the AI can be generic,

A generic AI framework may expose diplomatic interface, including private variables that are used to weight on diplomacy and functions that calculate AI response and attitude toward other AI or player, these functionalities are to be chosen or excluded by game as wanted.

Another example where the AI may be generic is path finding,
AI framework may expose different kinds of path finding, ex. hexagon based, node based or square based.

Another example where the AI may be generic is hierarchy and interaction of it's components,
ex. an AI framework may provide generic interfaces (not implementation) for various components such as economy AI, expansion AI, combat AI, unit design AI etc.
You provide implementation but without needing to reinvent naming convention and interaction.

Another example is espionage, a lot of it could be generic.

This also means making AI debugging tools upfront is possible.

---

You mentioned "We are not talking about learning AI"
Well what downside is there from such AI framework providing speech recognition module which would let AI player recognize your voice and act accordingly?
It's not something we see in games, but if you don't need to do it for every game separately then it's not a big issue.
speech recognition could recognize simple sentences such as "you're dead" or "I declare war on you", why not? is that hard?

This are just few examples,
My whole point is that, AI framework or generic AI can be generic in specific areas and may help in writing less of code, that is having reusable code that is known to work well, and be used as needed.

[Jorgen_CAB]

It might be a nice thought, but to be honest... the work of adapting what little code you have are probably going to be more resource intensive than writing it specifically for whatever game you are creating. Unless you are writing a series of similar games and can reuse part of the code.

When it comes to pathfinding that is not really AI.

[zebanovich]

I'm not into AI but from what I learned AI engine or frameworks are written in languages such as C++ while specific game logic is scripted, which is great for computation and doesn't consume as much resources since all the intensive works is delegated to the compiled generic portion.

This means if AI is not generic\compiled and is all scripted designed only for specific game then of course it would consume too much resources compared to generic compiled AI.

Yes, I don't think it could be possible for all around AI ofc. but rather specific one designed for 4X genre.
I didn't find such framework anywhere online, but I believe who ever comes up with such engine will make a good profit too.

[Jorgen_CAB]

I think you are conflating what scripted means... the game logic is NOT written in a script language and that is not what is meant with a scripted AI. The AI is written as part of the game which usually is either C++ or C# depending on the engine. So I don't get what you mean. A game such as this is pretty much written by ONE guy and the AI is written as part of the the game and game logic at the same time, it is a single player game.

I'm pretty sure you can't even detach the AI code from the game logic as it is part of the game logic in a sense, there are many parts of the game the player can't even control.

Ai is also not just ONE thing, AI is in play in many different ways and mechanics and not always connected with each other. Having a generic AI might even create more work than not. Who would create that AI and who would pay for it... this game is pretty much a one man show in terms of programming, pretty much?!?

What other games should it be used for?!?

[zebanovich]

What I meant to say by "game logic" is that scripted portion of the AI takes a game state and does the decision making,

I wouldn't be surprised if the AI is completely baked into the game, however how much of it is compiled varies from game to game, I'm pretty sure you can separate AI from game a lot.

The point of separation is not to make it more expensive but reusability and less work, initially it would take a lot of work for sure but later you have a working framework upon which you only need to do minimal work per game, and primary rationale for this is desire for better AI.

[Jorgen_CAB]

But... then again... reusing code or part of code is still something you can do. Relying on the experience to write a better AI in the future is also a part of that process. Knowing what you will need or what will work in the future in a generic AI structure I would question the validity on in general. I don't think you have proven a sufficient case this is even possible, just the notion of it does not really convince me that it is even practically possible.

Aside from reusing very basic things like pathfinding and previous experience of what works and don't in AI logic. I don't find how you can just graft on AI logic to a game without heavily alter how it works. AI is not a brain you can simply teach different mechanics that have varied input and output that do very different things to different games.

Especially complex games such as this one... programmer experience is probably the best transfer of AI programming, not some generic AI you likely will need more work to get working and probably in a suboptimal way.

In distant worlds all code except for the event system are compiled from either C++ or C# (I presume anyway any of those languages). So the AI is simply part of the game and the logic and part of the game structure in many cases. The high automation of the game is part of the AI and how it works. In many cases AI is not all that complicated either, it is just a set of parameters when the automation will do X or Y and maybe some randomization thrown in.

I would like to see someone writing a generic AI to, for example, Stellaris and Distant World controlling war. It would require two totally different set of AI as the mechanics are so far apart it might just be two completely different game types.

[zebanovich]

I would like to see someone writing a generic AI to, for example, Stellaris and Distant World controlling war. It would require two totally different set of AI as the mechanics are so far apart it might just be two completely different game types.

It certainly could not deal with so high level stuff as controlling combat.

I think of a generic AI as reusable classes and models such as binary trees, neural networks, perhaps individual neuron classes to able to size neural network as wanted, speech recognition module and similar low level stuff.

Therefore stuff which helps in making AI, not an actual AI.

When focused on 4X genre, it can be specialized only for the genre, by exposing specifics which are universal within genre.

[Jorgen_CAB]

I would like to see someone writing a generic AI to, for example, Stellaris and Distant World controlling war. It would require two totally different set of AI as the mechanics are so far apart it might just be two completely different game types.

It certainly could not deal with so high level stuff as controlling combat.

I think of a generic AI as reusable classes and models such as binary trees, neural networks, perhaps individual neuron classes to able to size neural network as wanted, speech recognition module and similar low level stuff.

Therefore stuff which helps in making AI, not an actual AI.

When focused on 4X genre, it can be specialized only for the genre, by exposing specifics which are universal within genre.

I'm sorry, you pretty much lost me... exactly what neural net AI would be of ANY help. Neural net AI is VERY difficult to train and use, any change and you need to retrain it. Very time consuming.

I don't see how any of this would help and I don't see who would do it and who would pay for it to use it?!?

To be honest I don't understand how it could be used and to what effect... I mean speak recognition.. this has NOTHING to do with AI for example.

I have actually done some neural net AI and trained it, some simple stuff. And even simple stuff is time consuming to make. Just a tiny change of the mechanic or either input our output means you have to entirely retrain the thing. Perhaps not from zero, but allot.

[zebanovich]

Following 2 papers outline how it could be used for map analysis and decision making:

https://www.cmu.edu/dietrich/sds/docs/g ... golman.pdf
https://scholarworks.unr.edu/bitstream/ ... _12862.pdf

I don't expect you to read it, it's not easy to grasp but my point is that neural networks are applicable for strategy games in some areas.

I'm not saying that NN *must* be used in game, but if it could improve the AI then having the base in AI framework would help in rapid development since you don't need to start coding from zero.

In one of your previous posts you said path finding is not AI, and now you say that speech recognition is also not AI, seems like your definition of an AI is very limited or that it only deals with AI player controlling his empire and nothing else.

One example where speech recognition could improve player XP is in game advisor, which instead of only talking would also listen to player questions and direct player to answer.
For example you could ask the advisor, which tech do I need for bombers, and the advisor would direct you to tech tree and highlight the tech, he doesn't necessary need to give vocal answer.

I don't know how expensive in term of development time such advisor would be but this certainly deserves to be called AI, and I'm sure most people would not turn it off as it is mostly the case with in game advisors.

The only limitation to all of this is time needed and testing, but that's the whole point of a framework to reduce the time and work needed and to have a well tested starting point that is known to work well.

[Jorgen_CAB]

I have actually read one of those before...

Did I not just say I have actually done some neural net programming?!?

While I agree that there could be SOME benefit to NN in a game, this game IS NOT suitable to it at this time. There are too many issues and would be WAY to resource intensive. These kinds of games also change or adds mechanics that will impact NN regulated AI in a much more impactful way. The training necessary is also VERY resource intensive as well. The more neural pathways you need it will be exponentially so.

I really don't see how you could EVER justify any sort of speech recognition into a game like this at todays level of refinement of these systems and the cost in resources involved, you just would not be able to do that.

Using NN for pathfinding MIGHT be doable, but you still get into the same issue, there are WAY more parameters to take into consideration on a game like this than a much simpler game such as DOTA or Starcraft for example. The other issue is also how the development cycle look like. To be honest a simple algorithmic approach is WAY less resource intensive considering the amount of people working on this game.

You have to consider that this game is practically and Indy game developed by pretty much two persons and programmed by one person!!!

In my opinion your ideas are simply not feasible for this game at this time, that is just the cold facts. In the future we might see more and more NN in games, but you will see them in way less complicated games first and then adapted to more games as the industry get more experience in it. Today it simply is not realistic to expect it.

[Radamanthe]

Using NN for pathfinding MIGHT be doable, but you still get into the same issue, there are WAY more parameters to take into consideration on a game like this than a much simpler game such as DOTA or Starcraft for example.

You’re wrong about this. DW pathfinding can only be a lot simpler than most games like Starcraft. Nothing is obtructing the way from point A to point B, which is the fundamental issue behind the term pathfinding in programming world, besides the fact that game mechanics can make the problem worse.

The pathfinding in DW has no such issue, it’s mostly "as the crow flies", especially when moving from one system to another. The only difficult part is the fuel needed to go to point B, and checking nearby systems as relays if the ships don’t have enough fuel. Nothing much more than that, even considering nebulas slowing it down. That’s not to say it is simple, just that it is an order of magnitude simpler than what the term pathfinding is all about in games since ages: obstruction.

Now, for the AI, and as I see the conversation, you’re right that a NN is not needed. In fact, it’s superfluous in 99% (if not all) of games. And for one main reason which is that training a decent NN costs far too much resources to be a viable way, especially for a game which is still under active development. I doubt CodeForce has such resources, and in fact, even Blizzard doesn’t have these resources for its products (the impressive Starcraft AI made by DeepMind is another story since DeepMind is… Google).

There is great confusion about AI in gaming world. Most people think we talk about the same kind of AI as NN AI, while it is definitely not. The term AI has historically been used in games even before the NN AI (ie. true Artificial Intelligence) came into the spotlight, hence the confusion. Games AI are simply… algorithms, not NN. Often complex, but algorithms. Something you can control and understand the behavior, unlike NN AI which even specialists don’t really understand how it works once it is trained (lot of interesting writings about it BTW).

To be honest a simple algorithmic approach is WAY less resource intensive considering the amount of people working on this game.

That resumes it all. Except that's less a question of the amount of people working on it than the amount of computing resources needed to train a NN AI.

[zebanovich]

Thank you Jorgen for conversation, we obviously don't agree but it's nice to have a talk on the subject with somebody.

I understand that small teams may not have enough resources for extensive AI research like MS or google, but also I don't think it's something too expensive if focused on specific genre of game.

My rationale is this, to make a good game you need a game engine, making one takes ie. at least a year which means there is reasonable cost involved, ie. at least 12 wages times count of people working on it.
But once this step is done, the game engine becomes reusable, you don't need to make a new one for every new game, you may improve it or change some things but you don't need to make a new one since you already have one.
And this way a lot of time is saved to make each subsequent game.

In similar way, some time may be spent to make an AI framework, initially this in same way takes time and resources but once done it becomes reusable and upgradable, you don't need to write an AI from zero for each new game, just like it's the case with game engines.

I think a reason why this is not usually the case is because to make a game, game engines are necessity while AI framework is not, so while game engine is necessity to make a game, the AI is made on the fly during game development, there is no necessity to waste additional time or research on AI upfront.

Another (very irritating) reason why is not more time spent on AI is because indirectly majority of players don't seek an AI that wins, nobody likes to lose a game and even less nobody likes a game in which player loses most of the time, players really want to win and if they don't they don't like the game, there is a chance that they won't play again if they lose.
Do you want to win when you play game?

Would you like DW2 or any 4X game if it would be extremely difficult to win? or if the game would be designed such that game wins most of the time?
I think this is regulated by "game theory" and "game design", which teaches that player must be able to win the game.
I have so far not heard of a game where game wins, except for chess engines where only world champions win.

Therefore conclusion here is that AI doesn't have to be too good, suboptimal AI is what market seeks indirectly.
Min maxing mad mans that play on highest difficulty and seek exceptional AI that will beat their ass are just a few people on the whole player market, so their opinion doesn't really matter.

Like I said I'm not into AI so I could be wrong, but I did write some models like you and automation stuff because I wanted to understand how experts make this stuff.
For me the most difficult thing was learning basics, but otherwise writing code and designing is not something difficult, it's all depends on existing skills.

[zebanovich]

There is great confusion about AI in gaming world. Most people think we talk about the same kind of AI as NN AI, while it is definitely not. The term AI has historically been used in games even before the NN AI (ie. true Artificial Intelligence) came into the spotlight, hence the confusion. Games AI are simply… algorithms, not NN. Often complex, but algorithms. Something you can control and understand the behavior, unlike NN AI which even specialists don’t really understand how it works once it is trained (lot of interesting writings about it BTW).

To be honest a simple algorithmic approach is WAY less resource intensive considering the amount of people working on this game.

That resumes it all. Except that's less a question of the amount of people working on it than the amount of computing resources needed to train a NN AI.

Hi, thank you for jumping in!

I don't know why NN is given so much focus but I'm definitely not advocating that AI should be driven largely by NN.
I only believe that NN could be used here and there if it would improve the AI.
Training it is the only real issue, how much computer resources it would use depends on what it should solve and how fast.

More of a problem with NN is data rather than computer resources, ie. in turn based game a usual turn count is 500-1000 turns, I don't think this would be enough data to train it, and I don't know if training it upfront would be possible since each game is unique.

[Jorgen_CAB]

Using NN for pathfinding MIGHT be doable, but you still get into the same issue, there are WAY more parameters to take into consideration on a game like this than a much simpler game such as DOTA or Starcraft for example.

You’re wrong about this. DW pathfinding can only be a lot simpler than most games like Starcraft. Nothing is obtructing the way from point A to point B, which is the fundamental issue behind the term pathfinding in programming world, besides the fact that game mechanics can make the problem worse.

The pathfinding in DW has no such issue, it’s mostly "as the crow flies", especially when moving from one system to another. The only difficult part is the fuel needed to go to point B, and checking nearby systems as relays if the ships don’t have enough fuel. Nothing much more than that, even considering nebulas slowing it down. That’s not to say it is simple, just that it is an order of magnitude simpler than what the term pathfinding is all about in games since ages: obstruction.

I did say that is "might" be useful here...

To be honest the game have allot more to pathfinding than just the things you suggested. There are two different method of travel, slow boating and hyperdrives, fuel, nebulas and jump range. The developers also want ships to jump between systems and not in a straight line from one system to another for some reasons (even if they can). So it is more like a node system rather than just straight forward.

I think it would be possible to use NN here, but not easy. The biggest issue I have is when and it the game mechanics change, you now have to retrain it. It "might" also have problem supporting modding as well, depends on the modding support and what you can do with that.

[Jorgen_CAB]

Thank you Jorgen for conversation, we obviously don't agree but it's nice to have a talk on the subject with somebody.

I understand that small teams may not have enough resources for extensive AI research like MS or google, but also I don't think it's something too expensive if focused on specific genre of game.

My rationale is this, to make a good game you need a game engine, making one takes ie. at least a year which means there is reasonable cost involved, ie. at least 12 wages times count of people working on it.
But once this step is done, the game engine becomes reusable, you don't need to make a new one for every new game, you may improve it or change some things but you don't need to make a new one since you already have one.
And this way a lot of time is saved to make each subsequent game.

In similar way, some time may be spent to make an AI framework, initially this in same way takes time and resources but once done it becomes reusable and upgradable, you don't need to write an AI from zero for each new game, just like it's the case with game engines.

I think a reason why this is not usually the case is because to make a game, game engines are necessity while AI framework is not, so while game engine is necessity to make a game, the AI is made on the fly during game development, there is no necessity to waste additional time or research on AI upfront.

Another (very irritating) reason why is not more time spent on AI is because indirectly majority of players don't seek an AI that wins, nobody likes to lose a game and even less nobody likes a game in which player loses most of the time, players really want to win and if they don't they don't like the game, there is a chance that they won't play again if they lose.
Do you want to win when you play game?

Would you like DW2 or any 4X game if it would be extremely difficult to win? or if the game would be designed such that game wins most of the time?
I think this is regulated by "game theory" and "game design", which teaches that player must be able to win the game.
I have so far not heard of a game where game wins, except for chess engines where only world champions win.

Therefore conclusion here is that AI doesn't have to be too good, suboptimal AI is what market seeks indirectly.
Min maxing mad mans that play on highest difficulty and seek exceptional AI that will beat their ass are just a few people on the whole player market, so their opinion doesn't really matter.

Like I said I'm not into AI so I could be wrong, but I did write some models like you and automation stuff because I wanted to understand how experts make this stuff.
For me the most difficult thing was learning basics, but otherwise writing code and designing is not something difficult, it's all depends on existing skills.

I think that AI in a game like this is super important, I also don't think that people in general don't want a good AI because they want to "win". I have seen no numbers in any way this is the case. You just would give the player advantages over the AI in those cases. Saying this seem just like a cop out for supporting your argument. The developers of Distant Worlds also told us, many times, that AI is one of their top priorities in the game. It is primarily a single player game.

In my opinion people would be happy if the AI actually where to give them some trouble, even when they learned the mechanics of the game. In the beginning people could just play on the easier settings. If all you want is win, why have an AI in the first place?!?

I don't think this argument for building some generic AI algorithms hold much water. I also fail to see how using some generic AI algorithms for the Stride engine (used for this game) would make that much sense. You realize someone would have to do that, someone would have to pay for it and then someone would need to adapt it to whatever game they are using. How do you suppose that would work. You just seem to imply that it can without any real objective examples how this would work or be economically sound.

Again, this game in particular is programmed by pretty much ONE guy... not even a team of programmers?!?

Who on their "team" would do this?

As @Radmanthe wrote above, not even the big companies have the money for these kinds of project. How do you expect smaller Indy companies to have the resources for such huge projects?

I'm not dismissing your idea as something that could be possible. Anything that would make AI better in games I would fully support. I just don't think this approach is economically viable in general, if it was... it would be done.

[Radamanthe]

I only believe that NN could be used here and there if it would improve the AI.

Well, that's not to say no game ever used NN (few advertised doing so, even back in the 80's). But that's very basic NN, really, and the results are not more convincing than even an average algorithm.

More of a problem with NN is data rather than computer resources

This is a computer resources problem because, precisely, you have to feed a tremendous amount of datas to efficiently train the NN.

ie. in turn based game a usual turn count is 500-1000 turns, I don't think this would be enough data to train it, and I don't know if training it upfront would be possible since each game is unique.

That's not a question of the number of turns to play a game, but the number of game sessions. Too few turns? Train many game sessions. Good analogy is Tic-Tac-Toe. That's what... 5 or 6 turns? If you want to train a NN to be unbeatable at this game (which is lame of course since any coder can easily make a perfect algorithm for this, but just for the experience), you'll have to feed it with hundreds, if not thousands of sessions at least (depending on the type of NN).

DeepMind's Starcraft AI was trained with millions of game sessions. After the first one, I doubt it was able to even make any action which made any sense (well, once fully trained, it effectively sometimes made actions which did not make sense in the first place... until it won ;), but that's probably the interesting story).

[zebanovich]

ie. in turn based game a usual turn count is 500-1000 turns, I don't think this would be enough data to train it, and I don't know if training it upfront would be possible since each game is unique.

That's not a question of the number of turns to play a game, but the number of game sessions. Too few turns? Train many game sessions. Good analogy is Tic-Tac-Toe. That's what... 5 or 6 turns? If you want to train a NN to be unbeatable at this game (which is lame of course since any coder can easily make a perfect algorithm for this, but just for the experience), you'll have to feed it with hundreds, if not thousands of sessions at least (depending on the type of NN).

DeepMind's Starcraft AI was trained with millions of game sessions. After the first one, I doubt it was able to even make any action which made any sense (well, once fully trained, it effectively sometimes made actions which did not make sense in the first place... until it won , but that's probably the interesting story).

I don't think it's applicable to 4X games, Tic-Tac-Toe, chess, Go and similar games are not complex, there is a strict set of rules and no unknown factors so writing a strong AI for them is much easier than for RTS or 4X games which involve unknowns and a lot of rules and mechanics.

Therefore running multiple sessions to train NN makes no sense to me, since every session is unique with unique problems.
An exception to this would be AI learning what ship designs to build against his opponent and similar stuff which is composed of strict number of factors, learning ship designs could be used across multiple sessions.

But waging war definitely could not.
Tactical combat could be trained, but in strategic combat and waging wars there are too many factors.

I think that AI in a game like this is super important, I also don't think that people in general don't want a good AI because they want to "win". I have seen no numbers in any way this is the case. You just would give the player advantages over the AI in those cases. Saying this seem just like a cop out for supporting your argument. The developers of Distant Worlds also told us, many times, that AI is one of their top priorities in the game. It is primarily a single player game.

In my opinion people would be happy if the AI actually where to give them some trouble, even when they learned the mechanics of the game. In the beginning people could just play on the easier settings. If all you want is win, why have an AI in the first place?!?

You seem to be assuming that game buyers are all 4X fans, a lot of players are new to genre and a lot of players are not seeking chess-like unbeatable game.

I've heard some game designers take this fact into account and design games so that they don't drive these group of people away from game.
For this reason I find it irritating that non-fans indirectly dictate how strategy should work.
You have surely heard of "the AI cheats" claim? that's one example of those newcomers to genre who don't even know what cheating really means. AI having bonuses is not cheating, cheating is when you use cheats such as instantly spawning a new fleet.
The worst thing is when they publish their disagreements on steam reviews, this further drives non-fans away.

I don't think this argument for building some generic AI algorithms hold much water. I also fail to see how using some generic AI algorithms for the Stride engine (used for this game) would make that much sense. You realize someone would have to do that, someone would have to pay for it and then someone would need to adapt it to whatever game they are using. How do you suppose that would work. You just seem to imply that it can without any real objective examples how this would work or be economically sound.

Again, this game in particular is programmed by pretty much ONE guy... not even a team of programmers?!?

Who on their "team" would do this?

As @Radmanthe wrote above, not even the big companies have the money for these kinds of project. How do you expect smaller Indy companies to have the resources for such huge projects?

I'm not dismissing your idea as something that could be possible. Anything that would make AI better in games I would fully support. I just don't think this approach is economically viable in general, if it was... it would be done.

I would really like to hear from various gaming companies how economical would be to invest into AI and even more how viable it would be to maintain a generic reusable AI, to what extent this would be possible, and whether this would improve it.

[Radamanthe]

ie. in turn based game a usual turn count is 500-1000 turns, I don't think this would be enough data to train it, and I don't know if training it upfront would be possible since each game is unique.

That's not a question of the number of turns to play a game, but the number of game sessions. Too few turns? Train many game sessions. Good analogy is Tic-Tac-Toe. That's what... 5 or 6 turns? If you want to train a NN to be unbeatable at this game (which is lame of course since any coder can easily make a perfect algorithm for this, but just for the experience), you'll have to feed it with hundreds, if not thousands of sessions at least (depending on the type of NN).

DeepMind's Starcraft AI was trained with millions of game sessions. After the first one, I doubt it was able to even make any action which made any sense (well, once fully trained, it effectively sometimes made actions which did not make sense in the first place... until it won ;), but that's probably the interesting story).

I don't think it's applicable to 4X games, Tic-Tac-Toe, chess, Go and similar games are not complex

That was a simple analogy against your statement about the fact that the number of game turns is "not enough data".

Therefore running multiple sessions to train NN makes no sense to me, [...]

That's how we train NN AI though, to beat the best human players.

[...] since every session is unique with unique problems.

That's precisely because each session is unique that the NN AI learns. I'm surprised you don't get it. You know, NN work more like human brain than you seem to think. Their limitation is that they have much less neurons, they don't live in our real world, have no corpse, etc. And above all: they're specialized, we are generic. There is no generic NN AI, that does not exist, and for a very long time. Probably for the best...