Colonization Minister secret revealed

After 12 hours of playtesting my AI I finally discovered why so many colony ships were hanging around loaded with no orders to plant and never getting orders. Orders are issued to a colony ship to plant by the minister when the number of colony ships built plus the number of colony ships scheduled to be built AND AT THE TOP OF THE QUEUE is greater than or equal to the number of colonies EXCLUDING HOMEWORLDS.

The test is made after each order is issued and the orders loop stops when the test fails.

This is why the colonization minister really sucks when you turn full AI on. I suppose it is a limiter on rampant AI expansion. There MAY be other tests such as how old a colony ship is, but my AI was building them so fast (10+54 colonies planted turn 30 with 38 ships waiting orders and more scheduled) that there was no way to tell. Also the rules for the AI when created as controlled by computer seem to be the same as for full AI on....

BTW my AI went bankrupt on turn 29 from building ships, with ships/bases at 89% of minerals production in maintenance and minerals usage deficit of 183k and no minerals left in storage. The colonies were spending all their time building units rather than mineral facilities. The ones who had yards were building ships/bases. Obviously that needs rework.....

[Andr�s]

Could you see the entire quadrant from the beginning? I think that changes the way AI colonizes.

All right, I was wrong. Two 30 turn test games reveal that the actual test passes if ALL ACTIVE ships plus ALL ships at top of build queues is greater than or equal to the total number of colonies EXCLUDING homeworlds. Bases do NOT count. I did NOT check to see if mothballed ships would count, but I seriously doubt it. If the test passes then the Colonization minister dispatches a colony ship to the best known colony given the needs of the empire. If the resource balance is good and not much tech is learned, then the highest priority is any known ruins colony. Otherwise the ruins are ignored and the best m/o/r planets are selected from the highest capacity working downwards until the resource balance would be very good given their expected production. Then it tosses in research, intelligence, military installation, construction yards, and resupply depots if they do not yet exceed their quotas. With only 70 of 1200 planets in my nexus central 17 systems colonized, I cannot determine what it would do next. But see my previous map posting in "really big games Version 2" thread where I describe what the AI colonization order OBSERVED was. That was for every planet 100/100/100 mild of the ones available in the nexus. I did not note what allocation was made for colony specialty since it did not matter.

What I did in the earlier tests today was omit all ships except colony from the build queue. I built them, yard bases, and units to see how fast a units only empire could grow. The allocation was as follows
Large Fighters - 10 @ 25kt builds 2/t for 5 turns
Large Mines - 10 @ 20kt builds 3/t for 4 turns
Large Troops - 10 @ 20kt builds 3/t for 4 turns
Large Weapon Platforms - 2 @ 600kt - builds 6t each for 12 turns
Total cargo 1850kt of 2000 for Cargo facility II, or 3000 for III.
Large Satellites - 2 @ 120kt - builds 2t each for 4 turns.

Total average per colony is a lag of 29 + 3 for cargo = 32 turns before productive facilities begin construction. All colonies do units without regard to how many they have produced of each type as selected by the construction minister. The build queue repeats colonizer/ mine layer/ mine/ fighter/ weapon platform/ satellite then some other ship/base about 8 times to entry 122. This gets the units quota scheduled for all new colony as soon as they plant. Shuffling them around is a chore for the Transports minister, but I doubt he is up to it. Mines WILL be picked up by the minelayers VERY slowly then laid on warp points VERY slowly. Troops WILL be picked up by troop ships VERY slowly. I did not have an enemy to use them on. Fighters WILL be picked up by carriers VERY slowly. Satellies WILL be launched on the turn after build. Obsolete weapon platforms will NOT be jettisoned when cargo is full. The AI REPEATEDLY schedules build of units on colonies with no space, resulting in NSA Messages, so that bug/feature has NOT been fixed when under player control with full AI on.

At turn 30 with 62 colonies I was 6 turns behind on units production. Checking back, I had units built for the number of colonies as of six turns ago. So I could keep it up forever if there were unlimited resources. I was utterly bankrupt at 30 if I built yard bases. But at turn 30 with NO YARD BASES and NO shipyards except on the 10 Homeworlds, I got the SAME number of colonies. I was deficit 24k/store 1106k minerals, and deficit 5k with 150k stored in radioactives. So I need some way to create a lag time when no units are built so that colonies will do their facilities instead of units. I have figured out how to do that and simultaneously solve the colony ship pileup problem. I will repeatedly schedule build of just a few attack ships, defense ships, and yard bases with NO colonies or units. After a half dozen repeats then I will schedule build of attack bases and defenses bases, repeated a couple times. Then I do the stellar manipulation, then go back to the top of the queue for colonizers, units, and support ships. I will have the support ships done between the ships and bases and immediately before/after the set to bring them up to quota.

Due to a feature of the AI Design minister working with the AI ship construction minister, you have to REPEAT the calls for any ship/base/unit that you really want built on the current pass through the loop. The reason is simple. When tech is learned that makes a design obsolete, the construction minister removes any scheduled of that design from all queues IF work has not yet started. But the loop has moved onward from the point at which scheduling occured, and there is no going back! Repeating a call in case tech is learned before work starts on an item is a temporary workaround. In the long run what is needed is for the design creation minister to hand an upgraded design to the construction minister and let him REPLACE obsolete entries instead of deleting them

When I get the bugs worked out of my build queue, I will post it along with the map9a I built with a nexus. But until I see the AI doing what I want, I am reluctant to post the map. This is the third big map I have put a few days work on and I want it to be the LAST for a LONG time.

Now it looks like the test is all colony ships plus all ships at top of build queue equals or exceeds number of colonies excluding homeworlds. I am beginning to wonder if that sucker is changing the rules on me depending on the turn number and total number of colonies/ ships/ yards....

I think I have figured out the construction queue usage in Construction_Vehicles :

Entry 37 Type := Colonizer
Entry 37 Planet Per Item := 10
Entry 37 Must Have At Least := 0

Build one colony ship for every colony, counting colony ships NOT yet built but queued up AT THE TOP of the queues. That TOP of queue is VERY important!

Entry 38 Type := Mine Layer
Entry 38 Planet Per Item := 80
Entry 38 Must Have At Least := 3

Build one minelayer for every 8 colonies counting ships NOT yet built but queued up AT THE TOP of the queues, and then for good measure, ALSO build up to 3 MORE if the number already BUILT does not exceed one per 8 colonies - IGNORING how many are in the queues, build at least one, but stop before 3 if building yet another would exceed one per 8 planets.

Entry 39 Type := Mine
Entry 39 Planet Per Item := 1
Entry 39 Must Have At Least := 1

Build ten mines for every colony counting mines NOT yet built but queued up AT THE TOP of the queues, and then for good measure, ALSO build just one MORE.

The TOP of queue MIGHT include ONLY those for which work has started already. I think it was put in because items will be removed from the queue if new tech makes them obsolete. So the count includes only those you can depend on having. Removing items from the queue when made obsolete would be MUCH better if instead they were REPLACED by the new design created when the design minister notifies the production minister that a design is obsolete. But that gets into issues such as the new design may require longer to build, if it is a unit the quantity may be different and there might not be cargo space, and you would also have to change the top of queue check to a full queue check. Obviously that would take more execution time. I think the extra execution time is not that important, but some threads about PBW talk about taking an hour or more to process a turn, so it MIGHT be important to others...

It seems the AI schedules items until all yard bases have at least one construction queue entry. For each queue entry (37 etc) it adds one and only one entry to the best yard available, then if not done it selects the next best yard until done or all yard bases have five or more turns work. (Note that yard components are INFERIOR to yard facilities, so usually all available colony yards are queued before the first yard base is selected.) It runs through however many queue entries are required to schedule five turns on yard bases UNLESS it hits a units production. If it hits a units production it schedules until the units entry is done or all colonies have five or more turns scheduled. If five turns are scheduled on all then it waits at that entry until it can be completed. If all the units entry can be scheduled it moves to the next entry. If it reaches five turns on all yard bases on an incomplete entry requiring a yard, then AND ONLY then will a colony without a yard get to build a facility, assuming that it has no units already scheduled. AFTER the vehicles scheduling is done, you may ALSO get queue entries added on colonies that have a yard if a facility UPGRADE is called for. So what I see is that any colony with a yard is allowed to build a facility ONLY if all the yard bases have an item scheduled at the start of a turn.

Furthermore, whenever new tech is learned that makes a queue entry obsolete, then the entry is removed UNLESS it has already started on a previous turn. This shortens the queues sometimes so that yard bases require scheduling more often than every five turns. I am not utterly certain about the five turns. It may be more or less, and it may be that the queue is done when one turn from build rather than when empty. It may also be that another is added when there is only one left. What I am CERTAIN is that the queues usualy have at least five turns work scheduled when I checked, and often there were three ships queued up.

"Yard bases" may include yard ships, but I have not built any so I do not know. It is obvious that a yard ship should only be queued if it has no orders to move. However I HAVE seen on my previous game checks to get scores at turns 30,45,60 that some AI had yard ships who were building something and recieving an illegal order to move.

If no "yard bases" are built, substitute "colonies which have built all facilities" i.e. Homeworlds and old colonies. Fortunately THEY have population bonuses to yard rate, which normally make them the next best available yard. So yards on newer colonies do not get called to build ships very often if no yard bases are built.

What this means is that the AI can BUILD YARDS on ALL its colonies, usually getting the facility per turn build rate enjoyed by players, but ONLY IF the AI designer gives up YARD BASES and YARD SHIPS. Furthermore, population transport may be a problem, because if the AI delivers enough to an optimal colony with a yard then it may exceed the yard rate of the homeworld. I have seen homeworlds as low as 1000k population, and delivery is made to exceed 500k. That issue I will have to explore in my next test game.

Now what I suggest is that the selection of best available YARD simply EXCLUDE any colony which has unused facility slots and ANY item already queued. Also do a facility build queue call when a YARD call selects the colony. If the queue is still EMPTY after the call, then ADD the ship to the queue, otherwise proceed to the next yard. It would be REALLY nice if changing units production could also be done at the same time. Basically I suggest that the sector inventory be checked to tally up the number of units of the specified type, INCLUDING all queue entries in all colonies in the sector (and not just at the TOP either). Divide the total by the number of colonies in the sector. If the average is less than the planet per item call, THEN schedule the unit. If the average equals or exceeds AND if there are unused facility slots on the colony, THEN do a facility build queue call. If the queue is still EMPTY after the call, then ADD the unit to the queue, otherwise proceed to the next colony.

I THINK the five turns queue ahead is useful because it gets the entries moved past ship/base entries so that units can be built. For some AI this might be critical. So keep it. Possibly this might be made an AI setup option rather than hard coded....

And while you are at it, add an option Rampant Colonization which lets the colonization minister dispatch every single colony ship as soon as it is built PLEASE! Make it a empire characteristic costing 500 points or something if necessary.....

I think these changes would be only a few hours work and would make the AI 1000 % smarter at building facilities, ships/bases and units. They would require only a few lines of logic and a few more function calls. It might be best to add another game setup option called DUMB AI for those who can not stand the challenge.

Anybody else want to lobby MM for these changes ?

[Mephisto]

> Now it looks like the test is all colony ships plus all ships at top of build queue equals or exceeds number of colonies excluding homeworlds.

So, in effect, the AI would not send out a colony ship when there was only 1 available, 0 building, if it had 3 settled planets?
I don�t know if this is correct. I have never seen such a behaviour, especially later this cannot be true. If the AI has 20 planets and only 2 colony ships ready and 3 building it will still colonise IMHO. Then again, I could not test it here at work.

>Entry 38 Type := Mine Layer
Entry 38 Planet Per Item := 80
Entry 38 Must Have At Least := 3
Build one minelayer for every 8 colonies counting ships NOT yet built but queued up AT THE TOP of the queues, and then for good measure, ALSO build up to 3 MORE if the number already BUILT does not exceed one per 8 colonies - IGNORING how many are in the queues, build at least one, but stop before 3 if building yet another would exceed one per 8 planets.

Mmm, my understanding is this:
Build one minelayer for every 8 colonies counting ships NOT yet built but queued up ANYWHERE in the queues, or at least up to 3.
So, if you have 8 colonies it would build 3 ships.
If you have 16 colonies it would still build 3 ships.
If you have 24 colonies it would still build 3 ships (�break even�).
If you have 25 colonies it would build 4 ships (25/8=3,125, round up = 4).


>I THINK the five turns queue ahead is useful because it gets the entries moved past ship/base entries so that units can be built.

IMHO it is not a good idea. The AI fills up the queues according to the build list of one of its �states�. So, if the AI is exploring, the AI fills up the queues with build orders according to the state �exploration�. If it now hits an aggressive enemy, it is still producing 5 (?) turns in the �exploration� state, switching to war production only after 5 turns have passed, a deadly mistake against a human player (don�t forget that after 5 turn only the production is switched but the ships/units must still be produced so it is more of a 8 turn shift).

> Now what I suggest is that the selection of best available YARD simply EXCLUDE any colony which has unused facility slots and ANY item already queued. Also do a facility build queue call when a YARD call selects the colony. If the queue is still EMPTY after the call, then ADD the ship to the queue, otherwise proceed to the next yard. It would be REALLY nice if changing units production could also be done at the same time. Basically I suggest that the sector inventory be checked to tally up the number of units of the specified type, INCLUDING all queue entries in all colonies in the sector (and not just at the TOP either). Divide the total by the number of colonies in the sector. If the average is less than the planet per item call, THEN schedule the unit. If the average equals or exceeds AND if there are unused facility slots on the colony, THEN do a facility build queue call. If the queue is still EMPTY after the call, then ADD the unit to the queue, otherwise proceed to the next colony.

This sound quite good. I would suggest to first to the facility build run and then the vehicle build run. Otherwise the AI cripples itself by not building any infrastructure.

[DirectorTsaarx]

quote:

---

Originally posted by LCC:
<snip>
Now I am just about SURE that the AI does not suffer this handicap. It is a PENALTY laid on lazy players by Aaron to discourage use of the managers!!
<snip>

---

Maybe it's due to the low/medium/high difficulty Ratings (or whatever the term is: easy/normal/hard?) for AI? IIRC, the different difficulty Ratings are supposed to affect colonization rate (among other things). Maybe the ministers default to easy or normal instead of hard AI...

quote:

---

Originally posted by DirectorTsaarx:
Maybe it's due to the low/medium/high difficulty Ratings (or whatever the term is: easy/normal/hard?) for AI? IIRC, the different difficulty Ratings are supposed to affect colonization rate (among other things). Maybe the ministers default to easy or normal instead of hard AI...

---

Outstanding point! I was running with medium difficulty - the default. Next time I will use high.

The attached construction queue goes bankrupt in minerals at turn 32 first LOR with starting resources high. There were 106 colonies and 54 ships, of which 29 were colony ships. The AI started building facilities and going LOR in all else. Unfortunately, many of the mineral worlds had their queues tied up in ship construction. But over the next 10 turns several facilities were built of each resource type each turn and eventually the AI would have come out of bankruptcy. An occasional ship or unit was also built, along with misc facilities. One more tweak and I think I will have it ok for economics - next time I will do high difficulty no bonus and default resource start.

Over 100 colonies at turn 30 is a very respectable rate, but I think I will go for 80 and not bankrupt. The Eee were 76 at turn 45 in my really big game. I figure a super Eee should be at least 50% faster....

You are absolutely right about the colony ship dispatch not being as stated. I finally cut my units per colony in half and stopped going bankrupt before turn 30. About turn 30-32 the queueing ahead multiple items finally stopped for some reason. As of turn 36 I had 105 colonies, 34 colony ships, 19 non colony ships, and 7 colony ships came out turn 37, leaving only two colony ships in the build queues. But 9 colonies planted to make a total of 10 + 104 planted. The rule had changed yet again! It is now colony/2 versus ships. So it is obvious that as the game progresses there is a rule set used by the colonization minister to limit your colonization rate by not dispatching ships when built. Note that you will probably NOT see this unless the number of ships built is significantly less than colonies planted and you have lots of colony ships under minister control. Now I am just about SURE that the AI does not suffer this handicap. It is a PENALTY laid on lazy players by Aaron to discourage use of the managers!! I have one more tweak to do to my build queue since I am about to go bankrupt at turn 38. Hopefully the next adjustment will get me past 60.

OTOH it occurs to me that possibly I have found a lost colony ship bug in the colonization minister since some of those colony ships are really old. Perhaps one of the beta testers would care to comment ? Running my savegame needs the map9a and the patch it does to add giant planets, plus the attached AI files in the next post - have to do a zip.

So far as scheduling facilities before ships/units goes, there is a problem. One of the first things an unprotected colony needs before ANYTHING else is a weapons platform. That has a higher priority than ANY facility. So you have to do the tedious cross checking I suggested from the ship minister....

[This message has been edited by LCC (edited 12 July 2001).]

Okay, here is my AI to date - as I say on the filename, it is flakey. I will delete this post in a week. You also need to go to the scenario/mod archive and download the map9a and install the patch for Giant planets. Create a directory Clays and unzip this file into it. Move the Clays empire file into the empires directory. Start game by load quadrant map9a. Select NO EVENTS. Select a 10 good planet high resources start. Select add only neutral AI empires low probability, so there is no hassle. Start and select no prompt for new colonies then turn full AI on and and just hit end turn. Clays spread like the plague! 114 colonies on turn 37! Unfortunately they go bankrupt.....

Edit - you also have to edit empire before you start game and select a shipset to use. I THINK you can do that. I am just using the Eee files renamed to Clays....

[This message has been edited by LCC (edited 12 July 2001).]

I just posted a file "Gobankrupt.zip" in the scenario/mod archive SE IV maps and scenarios. The game was started loading map9a. Events frequency none. High resources, home planet good, 10 planet start. Add Clays as empire 1 and select controlled by PLAYER using style from race. If controlled by computer the empire gets spread all over the quadrant despite 10 starting locations for empire 1 being given in the map, another BUG. Select player able to view scores of all players. Select neutrals only and low. Select high difficulty no bonus.
Start game and for Clays select no prompt for colony type, full AI on. Then change players to Clays controlled by computer and the neutral controlled by player. Watch the Clays score from the neutral. Ok, things I learned.

1) If you put space yards on colonies then they will ALWAYS produce a unit or ship once the yard is built, REGARDLESS of their yard rate and how many unused facility slots there are, until the empire goes bankrupt. So my suggestion to forego yard bases is turned 180 degrees around. Build LOTS of yard bases and yard ships, build NO yards on colonies, until this **** problem with building ships is resolved.

2) With yards on colonies the amount of time required to go bankrupt depends only on your starting resources and number/quality of planets start. The split between ships and colonies is up to you. You can have lots of colonies and few ships as in the previous post, or lots of ships and fewer colonies, as in the zip file saves. If you are going to go bankrupt anyway, it is better to have lots of COLONIES. The reason is simple. Once bankrupt the AI halts the ship/unit construction queue at the entry at which bankruptcy occurred. Projects already queued are not removed. But no new projects are started. See below.

3) The facility build is scheduled with facilities being run on colonies in the order specified in AI_planet_types ? The order for my files was Homeworld, mining colony, farming colony etc. What I see is that the homeworlds got to run a lot, but once the ship queues were emptied on the original homes, no further work was scheduled, because there were no facilities to be built. M/O/R was a puzzler :
Turn 34 production 111k/14k/13k storage 1k/316k/49k
Turn 75 production 319k/106k/42k storage 1k/1238k/79k
The minerals ran after homeworlds. Then only if the rads was deficit it got to run ? The orgs ran with whatever was left over.
The occasional project was also buiilt on other world types, which remains a mystery.

4) This is needed EVEN IF the changes in the previous post are implemented. What I suggest is that this IDIOT AI do what ANY sensible human player would do when facing minerals bankruptcy. Queue HOLD on all non mineral production scheduling until the minerals deficit is corrected. If a mineral colony is building something besides mineral extraction facilities, it comes AFTER the ones actually building them. ANTICIPATE the minerals bankruptcy by taking the total build RATE POTENTIAL of all colonies and ship/base yards as a sum, adding in the ship maintenance, multiplying the total by X turns (10 would be good, make it an AI_settings file entry) and subtracting the mineral resources in storage and X times the income total. If the result is a positive number, it is time to GET MORE MINERALS! Now check how many unused facility slots are available AND specified for minerals resource extraction within the next X facilities on ANY colony type. Take the total slots for each colony which can build WITHIN the next X turns and calculate the production they will generate. Sum the production POTENTIAL for all these colonies. Compare it to the deficit per turn calculated from the balance deficit above divided by X turns. If the deficit is greater, then it is time to PLANT MINERAL COLONIES. Tell the colonization minister to find the best known minerals potential planets. Dispatch colony ships to them from the idle colony fleet (see 8 below), and if not enough ships are built, then make a SPECIAL production minister call to build them, but NOT on the worlds which are supposed to build minerals!
AFTER that is done, then run queues on any other colonies that produce orgs/rads if those are in deficit, STOPPING when the balance becomes storage/X minerals deficit.
Then take queue holds off any queues which ALREADY have a UNIT scheduled, STOPPING when the balance becomes storage/X minerals deficit.
Then take queue holds off any colonizer production queues, STOPPING when the balance becomes storage/X minerals deficit.
Then run any ship/base production queues for which work has ALREADY started, STOPPING when the balance becomes storage/X minerals deficit.
Then run the remainder of the queues as usual, STOPPING when the balance becomes storage/X minerals deficit.

5) The AI WILL build monoliths! On rads worlds, with rads 900/t and monoliths 900/900/900 /t the AI chose the monolith! Of course the min/orgs at 1000 just built the regular facilities. So research stellar manipulation but not m/o/r above level 2. What is REALLY needed is for the AI to compare the TOTAL from all three to the single resource production and if significantly greater then build the monolith, same as a human would do.

6) The AI was squandering its puny residual minerals in storage doing retrofits. That might be smart for warships but was dumb for transports. Also the AI was trying to do illegal retrofits. What it should do is if the retrofit WOULD be illegal, then select the next most recent design until it IS legal, same as a human would do.

7) The AI is totally failing to anticipate the maintenance as a percent of production specified in settings. It should STOP scheduling and queueing up ships when the percent of maintenance WOULD exceed CURRENT resource levels. Anticipating future resources at the time of build is unnecessary since most ships take just a few turns to build.

8) Notice that there was a large fleet of idle colony ships in all the save games. So the number of colonies to be planted is NOT affected by the difficulty setting or player/computer controlled. It may very well be too **** complicated to deduce from just taking stats on planting. I tried, but without a detailed check including what had orders to plant but had not yet planted, I cannot tell from just watching the score from the neutral. I am NOT going to load 70 savegames to check it. Let it remain a mystery, and it is probably subject to change without notice anyway....

9) This AI build queue might NOT lead to bankruptcy for a long time IF the AI is able to make and keep trade treaties. But the other AI will put it in a world of hurt for planting so many colonies and eventually even the most tolerant partner will break, even if MEE is not reached first...

10) I am going to revise it to build lots of yard bases and yard ships and take the space yard facilities off the colonies. Then I will play test until I find what avoids bankruptcy and post. Basically you must have extended periods of time when few or no units are built, relying on warships and mines to defend the empire. I will keep regularly building mines and the initial ramp up with weapon platforms, fighters, satellites and eventually troops. The troops are important even if the AI never captures a colony because they increase happiness. What is needed is 15-20 turns of rapid expansion with units, followed by 40 turns of ship/base/mine production then cycle back to the start. The 40 turns can be accomplished only by planting colonies between ship/base calls so that there is a need to build more. Otherwise the entries are satisfied with no new production and it just wraps around to colonies and units again without getting facilities built. EXPAND OR DIE!!!!!

11) I am sending an email to MM pointing to this thread. Anybody with other minister complaints can post here and know it will be seen.

12) The attached queue file is the only change from before. Now I am going to get some sleep. Up over 24 hours again...