Site Searching Statistics Questions

[sector24]

You'll have to excuse me because I R not guud at math, but I have a few site searching questions.

1) If you use a site searching spell like Haruspex and don't find anything, does it have any effect on the chance of finding a site of another magic path? I would think that not finding a nature site would also eliminate the chance of finding a nature/astral site or nature/anything site as well. So as you eliminate more and more magic paths, the chance of there being a site in the remaining paths gets smaller and smaller. Is this correct?

2) Follow up question. If you search nature 1 and don't find anything, how does that affect your chances of finding a site at nature 2, 3, or 4? Presumably the greatest chance of finding a site is at level 1, but the better sites are rarer. Is this correct as well?

[lch]

The more you find out what's NOT there the higher the chance gets that something ELSE is there. If you already remote site searched a province and didn't find any sites at all, then unless it's farmland there's a high probability that something else is there. If you already found one or two sites, that could mean that those are all, though. It could be a magic site which is from a path unaccessible to you, like blood. Strong hints for magic sites are: unnatural scales of cold that can't be explained by or even contradict dominion influence can indicate that a water site is there, death means death (d'oh), heat can indicate fire and I think luck can indicate the presence of an astral site. This was really useful to search for magic sites more effectively for me. You should always search those provinces where you have found the least magic sites so far. Depending on terrain there's a higher/lower chance for some paths, too, check Edi's DB or I think somebody even made a list somewhere.

[Micah]

To elaborate on what Ich said: Sites are discovered based on a SINGLE path. The types of gems that a site gives is independent of the discovery path. For example, the clam fields site produces S gems, but requires W to find. The dying forest (produces 1N1D) is ONLY found by a nature search, death will not uncover it.

That being said, the vast majority of gem sites are found by searching the same path...W searches yield W gems, etc. Generally any off-type gems will be in addition to one or more gems of the searched path, but it's not always true.

Failed searches will generally mean that there is a higher liklihood that searching other paths will pay off. For example, with a 50% site frequency the chance of having 0 sites in a province (disregarding terrain mods) is approx. 6%, so empty provinces are somewhat of an exception, meaning it's proportionally more likely that sites are hiding in the unsearched paths. Likewise, only 6% of provs will have 4 sites, so a province that already has 3 sites in it is unlikely to merit further searches. (there are 4 potential slots for sites in each province, and .5^4 is how I'm getting the percentages here, the .5 will change based on site frequency). The higher the site % the more this will hold true.

As for levels, I generally do fine with manual searches, which cover mostly levels 1 and 2, but if you're looking for powerful discount sites you'll want to search to level 4, especially in astral and death. I'm not really sure how the site freq/terrain mods/site rarity/site income all combine in terms of how many gems you'll get per level from 1-4 on average though, since they're poorly documented from what I've seen, and my ability with that many probability interactions is lacking.

Higher level also isn't always better, most of the gem types have a level-3 search site that produces just a single gem.

[Baalz]

Micah said:
Failed searches will generally mean that there is a higher liklihood that searching other paths will pay off. For example, with a 50% site frequency the chance of having 0 sites in a province (disregarding terrain mods) is approx. 6%, so empty provinces are somewhat of an exception, meaning it's proportionally more likely that sites are hiding in the unsearched paths. Likewise, only 6% of provs will have 4 sites, so a province that already has 3 sites in it is unlikely to merit further searches. (there are 4 potential slots for sites in each province, and .5^4 is how I'm getting the percentages here, the .5 will change based on site frequency). The higher the site % the more this will hold true.

I don't think this is quite right. From what I remember about statistics the logic is like this. If you flip a coin 10 times and it comes up heads every time, what is the chance it'll come up heads the next time you toss it? Answer: 50% because the coin has no memory. Likewise, a province that has generated 3 magic sites has exactly the same chance of having a 4th as an unsearched province has of having at least one site if I understand how the sites are generated. You're right that a completely virgin province has only a 6% chance of having 4 sites, but a province that already has 3 sites revealed has a 50% chance of having a 4th with a 50% site frequency*.

That * is because there is another factor effecting your chances, which is again a bit counterintuitive. The more paths you've searched, the less likely you are to find more sites, regardless of how many you've found so far. As an example, consider a province that you've already searched in 7/8 paths. Assuming for a second that all paths had an equal frequency of sites, with a 50% site frequency the chance of there being at least one site there is (site frequency) * (chance of a given site being the path you haven't searched) or 0.5 / 8. Think about it this way, if you're casting aschric record you'll get the highest payout targeting provinces that have not been searched at all.

This is a little misleading though, as you usually don't care what your chances are that there are undiscovered sites, you care what your chances are of finding a site if you for example cast gnome lore. The thing is, from a statistical standpoint the paths are completely independent, if you search a province for astral it does not effect your chances of finding an earth site there regardless of whether you uncover an astral one. This is because what you really care about is "what is the chances a given province has at least one earth site?" which is not effected by the existence of astral sites - its 0.5 / 8. (disregarding the small chance that there are 4 non-earth sites, which complicates the math and doesn't change the chances too much)

[IndyPendant]

I'm not sure that's a correct analogy either, Baalz. (I'm not a statistician though, so I may very well need correcting.) The key difference, I think, is that the random chance has *already* been decided. Granted, if Dom3 checked that province each and every time a sitesearch spell was cast, there would be an equally slim chance each time of finding a suitable magic site regardless of how many sites have previously been revealed. But the game creates the sites randomly when the map is first generated.

Let's use your coin-toss analogy for the sites. I have *already* tossed the coin four times. In a random order, you check three of those tosses, and discover heads all three times. What is the chance that the fourth (randomized) pick will be tails? Well, let's see what options we have left:

HHHH
THHH
HTHH
HHTH
HHHT

So, from the options that are left--assuming a completely random pick order for the 'revealed' coins--the chance of the final coin coming up tails is *four times* the chance of it coming up heads.

Now, to bring it back to Dom3, that would mean that if a coinflip determined whether there was a site, then if you'd already revealed 3 sites, there would only be a 20% chance of a fourth site being revealed.

Or are the basic assumptions I've started with flawed?

[Lazy_Perfectionist]

Well, the problem is that I remember none of this math.

Hmmm... I think this would be a working model... Though a bit innacurate, fairly approximate.

Take four quarters, toss them, and put them down on a table. Heads is a site, tails, no sites. The table is the province. Now put a blank napkin on the coins that came up tails. No site here. For each coin heads up, write F, E, W, A, S, B, N, D, H on a napkin, representing each site path, including holy. Not looking, randomly take one of these napkins and put it on top of the coin, letter hidden.

To further complicate the model, you could write a number representing the level of the path. But for simplicities sake, we'll assume the site will only be searched with level nine remote site searches, e.g. Dark Knowledge, and ignore the site level.

One logic flaw with this model is the idea that sites of all paths are equal. Holy is notably rarer, for instance, because there just aren't as many sites.

[lch]

Baalz said:
I don't think this is quite right. From what I remember about statistics the logic is like this. If you flip a coin 10 times and it comes up heads every time, what is the chance it'll come up heads the next time you toss it? Answer: 50% because the coin has no memory.

This isn't like a coin flip though. It's more like for every province you have three cups (actually, nine) and one pea (actually, zero to four) under one of the cups. You want to find the peas. You can turn up one cup at a time. For a province where you haven't searched yet, that's three cups and lifting one of them is a 33% chance of finding the pea. If you already lifted one of them and did not find the pea, then you'll have a 50% chance of finding the pea if you try again for that province. This is better than trying another province where you only have a 33% chance of finding one.

This is, of course, a very simple model which isn't exactly like what we have in the game, so don't take it literally.

[VedalkenBear]

Ich has it correct, and there's a distribution that is used precisely for this issue; it is the hypergeometric distribution.

Consider the following. A province in ME, by default, has a 40% chance of having a site. Let us assume no terrain modifiers are in place.

The number of sites that are available to find is given by the binomial distribution. The probabilities are:

0 - 0.1296
1 - 0.3456
2 - 0.3456
3 - 0.1536
4 - 0.0256

(Incidentally, this means that the average number of sites in a province with a per-site chance of 40% is 1.6.)

Therefore, if you have an unsearched province, and you cast Gnome Lore on that province, the probability that you find an earth site is the probability that there are X sites, and that there is at least one site among those X. That is:

[P(1 site) AND (Probability that site is Earth)] OR [P(2 sites) AND (Probability that the first site is earth OR Probability that the second site Earth)] OR [P(3 sites) AND (Probability that first site is Earth OR Probability that second site is Earth OR Probability of third site is Earth)] OR [P(4 sites) AND (Probability of first site is Earth OR Probability of second site is Earth OR Probability of third site being Earth OR Probability of fourth site being Earth). Since AND is mathematically multiplication, OR is mathematically addition, and P1 OR P2 OR P3 ... is equal to 1-P1*P2*P3..., we have, assuming equal chances of each type of site in this province:

(0.1296)(0) + (0.3456)(0.111) + (0.3456)(0.210) + (.1536)(0.298) + (0.0256)(0.376)

= 0.1663

or an 16.33% chance of finding an Earth site in that province.

Now, what happens if you have an Earth site in a province, and then cast Astral Probing on the same Province? Then the probability is:

P(2 sites)*(1-P(that site is not Earth) + P(3 sites)*(1-P(neither site remaining is Earth) + P(4 sites)*(1-P(none of the three sites remaining is Earth).

Or

0.3456*0.111 + 0.1536*0.210 + 0.0256*0.298

= 0.0783

Or an 7.83% chance of finding an Astral site in a province that already has another Earth site, _provided you did not search for the Earth site_. If you did, then it is 0.0876, or an 8.76% chance.

I hope this makes sense. If anyone wants other probabilities (especially the way this all changes if you select a different per-site probability), just let me know.

[lch]

Okay, I think Baalz has trouble with the reverse scenario than I described: If I already found something, how big is the chance I'll continue to find something if I search more?

Baalz said:
The thing is, from a statistical standpoint the paths are completely independent, if you search a province for astral it does not effect your chances of finding an earth site there regardless of whether you uncover an astral one. This is because what you really care about is "what is the chances a given province has at least one earth site?" which is not effected by the existence of astral sites - its 0.5 / 8. (disregarding the small chance that there are 4 non-earth sites, which complicates the math and doesn't change the chances too much)

Those things are not independent. For every site searching, there are two things that come into the equation: One, how many sites there are in the province, two, how many you found already. If there are two sites in a province and you already found two, then you can search all you want but that won't make a new site pop up. That's because those sites are rolled for before the game starts. If they'd be rolled for when the site search is being made, you would be correct.

[Jazzepi]

I believe that the sites are all rolled up randomly by the generation mechanism for each map. When it is created all the sites are already distributed. That being said, let me try to give some insight as best I can. I'm not good with probability, Ich is, so hopefully he'll respond.

1) AFAIK, sites are only related to one particular magical type. A sight that produces both earth, water, fire, and astral gems will only be a site of one type. This means that searching all the nature sites to level 4 will not reduce the chances of finding an astral site that also produces nature gems.

2) It's useful to look at the # of sites in a distribution along the levels. There are more level 1 sites then level 2 so on and so forth with their being more level 2 sites then level 3 and more level 3 than level 4. Literally 95% of sites are level 1-3 which is why I always advocate that rainbow site searching mages take level 3, not 4, of a given magic path if all you care about is finding sites. You'd have to actually bring up Edi's open office spread sheet with every site in it, organize it by a given type, and then count the # of sites of a particular level to figure out the probabilities.

Suffice it to say, I don't consider a province fully searched until I've done so up to at least level 3. I would never site search, unless I was desperate, with a level 1 path mage. I would site search with a level 2 in multiple paths like EA Ten'Chi's flying capital only mages, or with a mage in 3-4 in a single path.

Hope that helps.

Jazzepi