Ah, thanks Patrick. I will have to explore that. Also thanks to wings7 to the Matrix games engine! Also something I will take a look into.
ADDIT: having had a look at those two engines (briefly) I think neither one is really suitable for my purposes. Cry seems to be an FPS high-graphics load engine, which I cannot see being practicable for a game intending to represent such massive amounts of geography.
The Matrix Engine I may actually download and play with, but I suspect its specific focus on a wargaming scale would not allow much for the 4x scale I have in mind.
@Matti,
No I don't have any experience at all. I'd enjoy the opportunity to help out on some projects that are not too far from my own interests.
My vision is still in the higher levels of abstraction phase, but basically it goes like this:
Turn-based isometric (possibly zoomabable/twistable, but no aspirations even to attain Civ5 level of graphical glitz) singleplayer historical 4x strategy game with possible provisions for coop and pbem, but no expectations of networked or multiplayer modes (and it is here I realize that Java's strengths offer little advantage to my envisioned game). Not having decided on a game engine / language yet, let alone started learning those tools, I'm still in the very beginning phase (which I've been stewing in for about 10 years, but it is finally time to get it moving forward). So, my vision for design is certainly incomplete. Related to this I recognize that it behooves me to be flexible and even opportunistic to "get my foot in the door" of game design. My first actual work, as you point out, might well be best spent helping some other group with their stuff, else getting a simple prototype that only reflects a portion of what I dream to eventually implement. As such, I'm trying to think in terms of a long-term developmental process, in which the initial game may simply introduce the game mechanics, themes and basic set of "rules," but cover only a small portion of the total envisioned subject area.
The Civilization series are certainly the most influential, but my main complaints with those games (and they have not got any better over the years) which I'd like to address:
1. Maps and scaling of entities, places, movement, etc. are ridiculous. The superordinate requirement for my envisioned game is that the engine must be able to support and run a game world of stupendous size and detail. Writing such game worlds, or developing algorithms to randomly generate them may take years, but the first step is to design an engine (pick an engine?) that can do it, and design a base game that can play it. If you try to translate say, the largest size Civ 5 planet you can generate, it would probably work out to be about the size of one U.S. state.
Basically, I want to design a Civ like game in which scaling (temporal and spatial) are as realistic as possible given the constraints of technology. Moreover, I would like the world(s) in which that game unfolds to be as detail rich as possible given the constraints of technology.
25 miles (~40km) across game cells is the number I arrived at as a "bare minimum" some years ago. If I come to the realization that the hardware and software will allow it, I'd happily plan on a scale that was double that size (12.5mile or 20km across game cells). Anything more fine-grained than that would probably be overkill for a grand historical strategy genre.
If you consider Earth's circumference is 24,901 miles (40,075 km), that works out to be just about 1000 game cells to accurately represent the east-to-west dimension of an Earth like planet and considering that significant portions of the extreme latitudes are pretty irrelevant to human history (in any direct sense, and modern nuclear missile sub lurking excuded

) then a game world that was 500? maybe 750 x 1000 cells? 750,000 cells each of which may have hundreds perhaps a couple thousand variables associated with it, and most of which would need to be editable during game play, not to mention the graphics; my reasons for keeping graphics relatively 'simple' may make more sense now?
The scale of what I have envisioned is probably beyond any game so far developed. This conception is not just a flight of fancy; I've been thinking about this during all the years I've played Civ and other historical and strategy games, and I'm convinced that if we want to usher in a new era of amazing historical strategy games, then such games need to be based on this sort of massive, and detailed model for representing the places in which history unfolds.
Only game I'm aware of that seem to be dealing with game worlds that are anywhere near that size is Minecraft. This is a major reason (along with Persson being filthy rich! [:D]) that I've so far leaned Java. I would love for someone who knows game programming to tell me with some conclusiveness if this vision is even tenable given modern hardware and software.
Related issues: there should be as much possible detail about each of these cells as possible and as much player flexibility in interacting with, shaping, and capitalizing on the resources of the world as possible. I don't know if I'd want to take it all the way to the level of Linnaean Taxonomy and modern Mineralogy/Forestry splitting of resources, but somewhere between that and the "Cow, Pig, Sheep, Fruit . . ." resolution of the Civ series is where I see it.
2. Obviously games must employ abstractions to be playable and to be developed within a timeframe that is feasible economically. That said, I want a grand society building game in which real biology, real psychology, real anthropology and possibly (eventually, probably) real history are the foundations for how the game plays. I say that being based on history is optional, but that doesn't mean that historically implausible or "sociologically-implausible" things should be afforded. All I mean by that is: initially the game may well need to have a "vanilla" flavor, in which there is little to no reference to the actual cultures and languages of Earth. This is because the real innovation I want to offer is in the way the game plays, not simply a more voluminous, precise, detailed or in depth replay of historical events. Not that I would be opposed to the game going in that direction, but I'd like the initial game to simply focus on portraying social, psychological, cultural and military events in the evolution of "a" society and its neighbors in as accurate and believable a fashion as possible.
The labor of creating the content to make the game "play through history" would likely be two to three times the scale of the basic game design itself (which is itself, likely many years of work). As such, I see the initial game having perhaps only one "playable" Civ and covering only a meager portion of human natural history (Neolithic to Dawn of Monotheism perhaps??, something along those lines). I may even opt for an even more abstract version of it, if generating the details for the AI societies proves to be just too much work and making them up would be quicker. Beta or Episode 2 or Expansions 1, etc., can always start to build the real Earth history into the game, but the main thing is to get a game that is sound, fun and compelling prototyped first.
3. If possible, gameplay should approximate real-time pausible and with adjustable time progression scales. An option to include a tactical level builder / combat interface to which play cuts from the strategic level map or when the player seeks to focus on her society at a more fine-grained level. I'm a big fan of the turn-based tactical battles games like "Temple of Elemental Evil" and "Jagged Alliance." Forge of Freedom took a crack at this, and it was good except that the random maps were a bit too random and homogeneous and the AI a bit predictable. I was impressed to learn that every tactical map in Rome Total War corresponds to one strategic map cell and when I realized that, including that in my vision was inevitable.
As far as development of the vision: I intend to do it alone, mainly because I'm the only one I can depend on for sure, but lots of other reasons. If and when it ever starts to take sail and I'm getting people coming to its website and maybe making Paypal donations for the pre-alpha free version, I'd consider starting to get help. But until that time, I feel that I need to do it alone in order to learn enough about all aspects of it that I can remain in the lead of it, no matter how far it goes, or on the other hand, if it just totally fizzles and goes nowhere, I will have no one to blame but myself.
The first game will be as simple as possible while still aspiring to the broad design goals described above. There are lots of other little things I have in my mind, but that is the main stuff.
So, what engine do you think is suitable for that?