Dice Analysis (using a z-test)

[tscott]

Like everyone else around here I have been getting mighty pissed off at these dice. I decided to run a statistical analysis on my my dice rolls. I performed a z-score significance test, which is what is often used in scientific studies. I used the dice analyzer tool to keep track of my dice rolls, and I am going on the assumption that the ideal stats given by the dice analyzer are correct (I will verify them when I have the time). So here are the results: With my current saved dice rolls I get a z-score of -1.92. What that means is that if we assume that the dice are random, there is only about a 2.7% chance that I would have rolled as poorly as I did. For reference, getting something below 5% (z-score of -1.645) is often the threshold used to indicate that our assumptions may not be correct. Almost 1900 dice "battles" were used in calculating this z-score, however, I will point out that the sample size is taken into account in this particular statistical test. This is by no means conclusive evidence of anything, but it does indicate that something may be amiss. If anybody is familiar with statistics, I would be more than happy to show you my calculations to see if I made any mistakes. Also, I have the formula in an excel spreadsheet, so I can update it regularly, or run the test on other people's dice analyzer stats.

[army of nobunaga]

send me a message with the details man.. what was the power of your test? After my test I dont believe this, I think there is a fault somewhere.

[tscott]

I'm not sure that determining the power of this test is useful or even possible. Technically the power would be 1 as the the null hypothesis is rejected (at the .05 significance level)! My calculations are on an excel spreadsheet, but I'm not sure how to upload it here so that people can see it. Basically what I did was figure out my proportion of dice won (.5534) and tested it against the proportion of dice I should have won (.5711). Note that the ideal proportion won will vary depending on the amount different battles you have done (3v2, 3v1, 1v1, etc.). The sample size then is the total number of dice thrown by me (2893). These numbers should give you the z-score of -1.92. If I can figure out someway to get the spreadsheet up, you can double check that I calculated the proportions correctly.

[army of nobunaga]

I use google doc.s .. easy way to publish spreadsheets. I dont think with a power of 2893 you should have gotten a 1.92 z if you did, then either 1) you misscounted somewhere. 2) the dice are not acting in a random behavior.

[tscott]

I'm not sure your using the term power correctly. The power of a test is a proportion that tells us how likely we are to reject the null hypothesis ASSUMING that the alternative hypothesis is correct. It is not really used in this situation. The 2893 is the sample size; that the dice aren't acting randomly is exactly what I am showing evidence for! I will get the calculations up on google docs. Is there a way to make it available to everybody? Or do I have to email it to you?

[Doc_Brown]

I don't have an extensive statistics background, though I do have a math degree, so I can usually follow the mathematics pretty well. One thought on looking at what you wrote here in the forums: You said you had 2893 throws with an expected result being .5711. I'm assuming that this is a weighted average of the expected wins from each type of throw (3v2, 3v1, etc...). I think you may be better off calculating your z-score for each type of throw individually, because I suspect you have very few 1v1, 1v2, or 2v2. A small sample size for these values may be enough to skew your results a good bit.

[army of nobunaga]

go to google, click google docs, upload sheet, publish to general population.. pretty easy

[army of nobunaga]

I don't have an extensive statistics background, though I do have a math degree, so I can usually follow the mathematics pretty well. One thought on looking at what you wrote here in the forums: You said you had 2893 throws with an expected result being .5711. I'm assuming that this is a weighted average of the expected wins from each type of throw (3v2, 3v1, etc...). I think you may be better off calculating your z-score for each type of throw individually, because I suspect you have very few 1v1, 1v2, or 2v2. A small sample size for these values may be enough to skew your results a good bit.

I have to agree that there is something overlooked because after only 250 dice pairs, my analysis showed a pretty bellcurve. at a 1.92 z, I would not have had a bellcurve. I have to run atm, Ill think about this though.

[tscott]

Hopefully this works: http://spreadsheets.google.com/pub?key= ... utput=html

[Doc_Brown]

As I said earlier, I don't have a lot of statistics background. Can you give me more details on how you compute the z-score? I realize that it's the number of standard deviations above or below the mean. How are you computing the standard deviation in this case?

[Georgerx7di]

In about 1 in every hundred turns I get these really bad dice. The odds of them being that bad are about 1%. That's ridiculous, that should never happen. How can they call the dice random.

[AAFitz]

In about 1 in every turns I get these really bad dice. The odds of them being that bad are about 1%. That's ridiculous, that should never happen. How can they call the dice random.

Well, technically, they should happen about 1% of the time. Technically.

I do have to admit ive seen far more 5s and 6s losing to ones lately, but I honestly think Im just looking for it. I find if I dont focus on the bad rolls too much, the game goes along fairly randomly. If I look for bad rolls, then they all seem impossible.

[army of nobunaga]

-how do you not have any ties in 3 vs 1 and 2 vs 1 ? Maybe i misunderstand.
-did you ever confuse dice roll with number of armys? (simple mistake)
-I assume you have SAS or something like it? I think you will need to do a 18 input Analysis of Covariance to get a better predictor.

Man Im tired Ill think on this tomorrow. I know im overlooking something. If you are right there is something wrong with the dice. Im pretty sure you have to treat all 18 outputs separate and compare them to the correct (predicted) 18 outputs to get a better analysis. You are adding up the dice at the end to get a z score and I believe that weighs them (all the various outputs) the same when they have different 1 to 1 weights.

[army of nobunaga]

in other words its the 2 vs 2 throws that are messing up the z score, Im not saying the data are wrong, just saying at the end you treat the 2vs2 the same weight as the other throws. And with even distribution that would throw your z score out of wack.

[Doc_Brown]

-how do you not have any ties in 3 vs 1 and 2 vs 1 ?

You can't have a tie when the defender only rolls a single die. The only possible outcomes are: 1) Defender loses 1 (attacker wins) or 2) Attacker loses 1 (defender wins).

[Doc_Brown]

in other words its the 2 vs 2 throws that are messing up the z score, Im not saying the data are wrong, just saying at the end you treat the 2vs2 the same weight as the other throws. And with even distribution that would throw your z score out of wack.

Actually, I think he is getting an effective weighting. The W and L columns are the total number of armies lost from a given set of rolls. Since he only rolled the 2v2 14 times, it only contributes 11 of the 1601 total wins and 17 of the 1292 total losses.

Now, I may not know the standard statistical approaches, but I do know simulations. I pulled up a Monte Carlo simulation in matlab and rolled 986 3v2 battles 5000 times. I calculated the total armies killed and compared that to the ideal number (1064.1) and drew a histogram of the difference between the actual and expected values which had a nice normal distribution. The OP got a difference of -27.1 which I get at the point (x-mu) = -1.06sigma. Running 820 3v1 simulations and drawing the appropriate curve, I can pick the OP's result of a -20.9 difference off at about the -1.57sigma position. I don't think there's enough test cases to worry about the other values. The combined results for these two cases falls approximately at -1.59sigma, which has a likelihood of about 5.6%. So it is an unlikely result, but maybe not quite so much as originally suggested.

[tracedout]

In about 1 in every turns I get these really bad dice. The odds of them being that bad are about 1%. That's ridiculous, that should never happen. How can they call the dice random.

Well, technically, they should happen about 1% of the time. Technically.

I do have to admit ive seen far more 5s and 6s losing to ones lately, but I honestly think Im just looking for it. I find if I dont focus on the bad rolls too much, the game goes along fairly randomly. If I look for bad rolls, then they all seem impossible.

holy shit dude

[jrh_cardinal]

I'll do a z-score if you want, but I can tell you I'm winning slightly more than I should be since I got the dice analyzer (last week). Out of 2770, I've won 1727, actually a good bit more than I should.

Whoever was asking how to get the standard deviation (Doc Brown I think)
These tests we're are doing are officially t-tests. In layman's terms you have to estimate the standard deviation because you don't know the population's standard deviation.

[army of nobunaga]

I couldnt sleep last night,and thought about this. His power for all the throws isnt high enough. Avery type of dice win/loss combination needs to have about 300 data apiece. I am right.

[army of nobunaga]

sorry dont mean to be short, running late for my haircut.

he needs more 2-2 through 1-1.

promise. When all those throws get to be somewhere over 100 you will see it even out. I think a Anacova, would work better, but it would still be out of wack with these data

[Doc_Brown]

Now, I may not know the standard statistical approaches, but I do know simulations. I pulled up a Monte Carlo simulation in matlab and rolled 986 3v2 battles 5000 times. I calculated the total armies killed and compared that to the ideal number (1064.1) and drew a histogram of the difference between the actual and expected values which had a nice normal distribution. The OP got a difference of -27.1 which I get at the point (x-mu) = -1.06sigma. Running 820 3v1 simulations and drawing the appropriate curve, I can pick the OP's result of a -20.9 difference off at about the -1.57sigma position. I don't think there's enough test cases to worry about the other values. The combined results for these two cases falls approximately at -1.59sigma, which has a likelihood of about 5.6%. So it is an unlikely result, but maybe not quite so much as originally suggested.

I cleaned up my simulation during lunch this morning and took out the approximation part. As before, I'm only considering the cases with 3 attacking dice since they're the only ones that have sufficient rolls for any meaningful analysis. I ran a Monte Carlo simulation using the same number of rolls of 3v2 and 3v1 as the OP was and calculated the total number of defending armies killed. The standard deviation (with 5000 runs) was 28.74. The OP ended up killing 48.045 fewer armies than would be expected, which has a likelihood of 4.7%. I'd say it would be worth collecting more data for another few days or a week and see if the numbers start to average out.

[army of nobunaga]

-how do you not have any ties in 3 vs 1 and 2 vs 1 ?

You can't have a tie when the defender only rolls a single die. The only possible outcomes are: 1) Defender loses 1 (attacker wins) or 2) Attacker loses 1 (defender wins).

Yeah I figured this out. At first I did conceptualize it. But I thought about his during my haircut. Ill bet a premium membership donation that once he records more 2vs2 through 1vs1 throws this will become less deviated.

[tscott]

Thanks for everybody's comments. i will certainly be updating the numbers as I get more rolls in. A few thoughts though: I had initially considered the low numbers of 2v1, 1v1, etc. throwing off the calculations, but I convinced myself (albeit intuitively) that my sample is all dice rolls, and that it shouldn't matter of what variety of dice battle they come from. Now I know intuition can be wrong, especially in statistics, so to me the jury is still out on this matter. But I will update the spreadsheet so that it will show a z-score for each type of battle. Another point which seems to be coming up is the issue of sample size (I don't mean to be rude armyofnobunaga, but what you keep referring to as power is actually sample size. The power of a test is a proportion, and thus will be between 0 and 1. Although it is related to sample size, it is somewhat of a different concept and not really applicable to what we are doing here). A large sample size is certainly better, but the z-test takes sample size into consideration.

@jrh_cardinal: You said that you thought we should be doing a t-test instead. I have to disagree though because we do know the true standard deviation and are not estimating it. With the sample size as high as it is, though, it shouldn't make much of a difference.

[Crazyirishman]

Are you sure you can use a z test, the dice might not be normally distributed? Or does the Central Limit Theorem kick in because the sample size is grater than 30.

[iamkoolerthanu]

Thanks for everybody's comments. i will certainly be updating the numbers as I get more rolls in. A few thoughts though: I had initially considered the low numbers of 2v1, 1v1, etc. throwing off the calculations, but I convinced myself (albeit intuitively) that my sample is all dice rolls, and that it shouldn't matter of what variety of dice battle they come from. Now I know intuition can be wrong, especially in statistics, so to me the jury is still out on this matter. But I will update the spreadsheet so that it will show a z-score for each type of battle. Another point which seems to be coming up is the issue of sample size (I don't mean to be rude armyofnobunaga, but what you keep referring to as power is actually sample size. The power of a test is a proportion, and thus will be between 0 and 1. Although it is related to sample size, it is somewhat of a different concept and not really applicable to what we are doing here). A large sample size is certainly better, but the z-test takes sample size into consideration.

@jrh_cardinal: You said that you thought we should be doing a t-test instead. I have to disagree though because we do know the true standard deviation and are not estimating it. With the sample size as high as it is, though, it shouldn't make much of a difference.

What is the true standard deviation then?