Saturday, March 5, 2011

Lies, Damned Lies, and Statistics.


I’ll be using this project as an excuse to start learning and applying statistical methods.  I’ve had a bit of exposure in school, but lack much real life experience, so the methods aren’t really at my fingertips.  Riflery is a prime place to use statistics as every practice session can become a data gathering exercise.  As I go through this, I’ll make every attempt to speak in English instead if Math, and go step by step.  I assume most shooters have no real statistical expertise.

I had always believed that the quote used as a title of this section was a Mark Twain line.  It turns out that not only did he not originate it, but he attributed it to someone who didn’t say it at all.  With fact checking like that, no wonder he hides behind a pseudonym.  The only information historians have been able to dig up is that it originated somewhere in the late 1800s.  I’m taking it upon myself to assume the origination date was 1877, the same year my trapdoor rifle was made.  I may be wrong, but there is a statistical possibility that I’m correct.  Ha!

OK, why statistics?  The whole idea is to use a little math to tell us something that isn’t obvious just by looking.  For instance, look at the 2 target groups that were posted previously.  They both look pretty similar.  By measuring the overall spread of the groups or measuring the smallest circle that will hold each group, we can tell that one is somewhat smaller than the other.  The question is, are the two groups so different that one ammo is better than the other, or are the two essentially the same and it was just raw luck that one group was a tad smaller?

Most basic statistical tests compare one set of values to another to see if the averages of each group are different.  Unfortunately, on a rifle, I can adjust the average to anywhere I want.  The two groups we’re looking at were centered over 12 inches apart, but with a tweak of the sights I could have put them right on top of each other.  The  feature I’m interested in is size of the group, or to say it another way, the distance of each shot from the center of the group.  The number for the average distance from each shot to the center of the group is the Standard Deviation.  If you check it out in a formula, there is some trickery to make it all come out positive and a small correction for the fact that the group is a small sample of all the shots using that ammo, but still, that’s the gist of it.  So now, we can calculate the Standard Deviations for the 2 groups and compare them.  I am splitting the measurements up into horizontal (X) and vertical (Y), so we’ll have 2 comparisons.

Horizontal:
Stand. Dev for Remington 300Gr X=2.03
Stand. Dev for Ultramax 405 Gr X=1.76

Vertical:
Stand. Dev for Remington 300Gr Y=1.75
Stand. Dev for Ultramax 405 Gr Y=1.97

You can struggle through these calculations by hand with a text book, but I recommend a spreadsheet program like Excel.  It has all the statistical formulae programmed in, but it won’t tell you when you should or shouldn’t use them.

So how do we compare these numbers?  The plan is to take the “F ratio,” which is just squaring the standard deviations and dividing one by the other.  For the numbers above:

F horizontal= 2.03 X 2.03 / (1.75 X 1.75) =1.32
F Vertical= 1.97 X 1.97 / (1.76 X 1.76) = 1.26

Then we check out a chart of how likely it is to get these numbers.  Again, I’m going long hand by checking a table, but Excel can do it all automatically with the “Ftest” function.  On the table, I get 2.25 and 2.17.  Since both of these are larger than our 1.3ish numbers we have no real certainty that the ammo is any different and any group differences were likely just chance.

So what now?  Well, we can try some other ammo.  If we think there is really a difference between the two we can redo the test with an attempt to reduce variation from other things.  For instance, if a better sight and a smoothed trigger can give me tighter groups, then maybe a retest of the ammo will be able to show a real variation.  Also, an increase in sample size, say 50 rounds of each instead of 20, will make it mathematically easier to show a difference, if one exists.  Common sense wise, I’m shooting an 8” when I should be shooting 3” or so.  I don’t think chasing this small difference is worth my effort right now.

No comments:

Post a Comment