Statistics for differentiating between groups?

[sclancy27]

Statistics is what your heart surgery will be based on. Such snide remarks ignore the fact that nearly everything that benefits your life was developed and applied through statistics.

You can go through life without understanding much about math, physics, biology, etc. And yes, statistics, but you would be a fool to denigrate any of them. They are all part of what allows you to live as you do.

Yes. Just because people watch other idiots butcher the use of statistics doesn't mean that proper application and modeling is wrong. If you asked a simple probability question 99.9% of people would get it wrong.

 

[grizzly63]

Yes. Just because people watch other idiots butcher the use of statistics doesn't mean that proper application and modeling is wrong. If you asked a simple probability question 99.9% of people would get it wrong.

"Probably"

 

[yakimanoob]

Closing the loop on this:

I was able to get my hands on a copy of Frank Grubbs's 1964 Statistical Measures of Accuracy for Riflemen and Missile Engineers, which is extensively referenced in shooting statistics works but out of print and quite difficult to find. Thankfully the Interlibrary Loan system came through for me and the University of New Orleans library had a copy. Anyway, Grubbs lays out the test I was looking for.

Say you shoot two groups, A and B, each with n shots and each using a different load. Measure the (x,y) coordinates of each shot and enter them into 4 columns in excel: the x coordinates and y coordinates for A, and the x coordinates and y coordinates for B. For each column, compute the variance using the VARP() function (not the more commonly used VARS() function). So you now have four descriptive statistics: the horizontal variance for group A, the vertical variance for group A, and so on.

The sum of the horizontal and vertical variances for a given group is called that group's radial variance, and the ratio of radial variances follows an F distribution with 2n-2 degrees of freedom. Thankfully, the F-statistic is easily obtained in excel using the F.INV() function. If you shoot 5-shot groups and want to use a 95% confidence level, for example, you can get the F-stat with =F.INV(0.95, 8, 8) since 2n-2 = 8.

If the ratio of the radial variances (for example, the radial variance of group A divided by the radial variance of group B) is higher than the F-statistic, then the group in the numerator of the ratio (group A in this example) is significantly less accurate than the other group at the given confidence level. If the ratio is less than the F-stat, it should be assumed that any difference in group size between the two groups is due to random chance (i.e., we can't say with confidence that one load is more accurate than the other).

I'm working on a full write-up of this method—let's call it the Grubbs Test—with real-world examples and a separate document with the statistical justification from Grubbs's text for those interested.

 

[Elktrack]

Yes. Just because people watch other idiots butcher the use of statistics doesn't mean that proper application and modeling is wrong. If you asked a simple probability question 99.9% of people would get it wrong.

You’re a 100% right

 

[yakimanoob]

A few notes on the Grubbs Test:

• we need remarkably few assumptions for the test to be valid -- only that the horizontal dispersion of shots within a given group follows a normal distribution and, independently, the vertical dispersion is also normal. They don't have to be the same (i.e., the "true" population distribution need not be circular), they just each have to be Gaussian, which is a fairly safe assumption if your rifle is in decent working order and your technique isn't systematically biasing your shots in one direction or another.
  • a little further digging and some help from (much) more knowledgeable folks revealed that Grubbs is indeed assuming a bivariate circular normal in this test. Almost all texts I can find on this subject that reference actual large-sample data indicate that a properly working weapon should pattern according to a circular normal distribution, but it's obviously worth noting the assumption.
  • UPDATE: it's still up in the air if equal variances between the x and y coordinates is a necessary condition for this test to be valid. I'm still seeking advice on this, but in the meantime I'll be conservative and assume the underlying distribution needs to be circular normal.
• you can measure the x and y coordinates from any point of reference you please, so long as you're consistent within each group. You don't even need to be consistent from one group to another. If you want to choose chaos and measure from a random point on your target for each group, go for it. You do you.
• units do not matter - you can measure in inches, millimeters, MOA, hands, light-nanoseconds, Smoots, whatever. Just be consistent within and across all groups.

 

[ImBillT]

I need to read this tonight. These are the types of questions we should be asking!

I got involved in a similar, but simpler, conversation a year ago and frankly could not remember enough from my stats class, nor was I able to find my stats book.

 

[yakimanoob]

Most stats books don't directly address this kind of thing. To be fair, the analysis gets quite complex if you don't assume that the underlying distribution is circular Normal, and to my knowledge there simply are no reliable methods available to analyze small samples if you also give up the assumption of Normallity.

I'm hoping to have the full write-up done within the next couple weeks, but my work schedule is pretty tight from here until late May, so we'll see what I can manage.

 

[ImBillT]

Statistics are greatly under appreciated by the benchrest shooters at my local range, but I must admit that their vast experience(10k shots per year for many decades) combined with excellent equipment makes their job a little easier. It’s interesting to see them shoot a rifle that is capable of consistently shooting down in the .100” range, and shoot a group with .5” of vertical stringing, or horizontal stringing, or “two grouping”, and then tweak the load exactly into line. On the other hand, I’m not sure anyone could do the same when examining groups from a rifle that AVERAGES 1MOA. Also, I’ve seen the same guys(benchrest hall of fame members) shoot two groups that both look great to me and declare that one is better than the other, while I’m not at all convinced that they could not just as easily have shot the second group with the first load. Some proper stats would help them out!

One thing I’ve noticed, that popped up in a post on the first page is the word “round”. Most of the benchrest guys at the range, if you ask them what they’re looking for when they’re tuning will say “round groups”. One of the previously mentioned statistical methods mentioned that a gun functioning properly should shoot circular normal groups. While I’m not sure that what a benchrest shoot calls a round group should be applied to rifles that are barely capable of shooting .5-1 MOA and using mediocre ammo, I still find that to be an interesting coincidence.

Still looking forward to actually reading this tonight.

 

[ImBillT]

Most stats books don't directly address this kind of thing. To be fair, the analysis gets quite complex if you don't assume that the underlying distribution is circular Normal, and to my knowledge there simply are no reliable methods available to analyze small samples if you also give up the assumption of Normallity.

I'm hoping to have the full write-up done within the next couple weeks, but my work schedule is pretty tight from here until late May, so we'll see what I can manage.

I’m convinced that something about shooting follows some sort of normal distribution. Picking the right thing to measure would be key.

Brushing up on my stats could point me in the right direction regardless of whether it was addressed directly. I need to just order a textbook off eBay.

I’m confident that someone here will get it reasonably figured out long before I have time to figure it out on my own as I have too many irons in too many fires, so I really like this thread. My work schedule is gonna be pretty tight from now to Oct.

 

[yakimanoob]

It’s interesting to see them shoot a rifle that is capable of consistently shooting down in the .100” range, and shoot a group with .5” of vertical stringing, or horizontal stringing, or “two grouping”, and then tweak the load exactly into line.

This is the kind of black magic that can only be done by true experts with a crap ton of experience, and it's just downright cool.

 

[std7mag]

I really hate to sound like a Debbie Downer, but....

For any testing to be truly relevant, the conditions of the testing needs to be controlled.

And from a firearms standpoint (or Howitzer barrel for that matter), it just won't happen.

As per everyone that asks for that "magic load".
This load works in MY rifle! YOUR rifle is different!

Add in things like atmospheric differences. Yeah, I know about standard atmospheric conditions. But unless you're in a controlled underground bunker, it changes.

Then there is powder/cartridge temp.

Was the humidity level consistent when charging the powder?

Oh, and the barrel/chamber.

Even IF the chambers were reamed with the same reamer, tooling wears.
Ditto the tooling for the rifling in the barrel.

Then you start shooting.

The throat starts erode beginning with the first shot.
Also you have carbon and copper build up.
And to top that off, the bullet traveling down the barrel will to some degree smooth out some of the markings left by the tooling that made the barrel to begin with.

Then, once you shoot your group.
How are you going to get the barrel to the exact same condition that it was for the next test group?

Statistics is fine, but for this pursuit, there are just too many variables to make it actually meaningful.

 

[BrentD]

Thankfully, you are not Debbie. You are just wrong.

 

[yakimanoob]

I really hate to sound like a Debbie Downer, but....

For any testing to be truly relevant, the conditions of the testing needs to be controlled.

And from a firearms standpoint (or Howitzer barrel for that matter), it just won't happen.

As per everyone that asks for that "magic load".
This load works in MY rifle! YOUR rifle is different!

Add in things like atmospheric differences. Yeah, I know about standard atmospheric conditions. But unless you're in a controlled underground bunker, it changes.

Then there is powder/cartridge temp.

Was the humidity level consistent when charging the powder?

Oh, and the barrel/chamber.

Even IF the chambers were reamed with the same reamer, tooling wears.
Ditto the tooling for the rifling in the barrel.

Then you start shooting.

The throat starts erode beginning with the first shot.
Also you have carbon and copper build up.
And to top that off, the bullet traveling down the barrel will to some degree smooth out some of the markings left by the tooling that made the barrel to begin with.

Then, once you shoot your group.
How are you going to get the barrel to the exact same condition that it was for the next test group?

Statistics is fine, but for this pursuit, there are just too many variables to make it actually meaningful.

All the variables you’ve mentioned are simply the sources of variance, and in no way do they invalidate the statistical methods being discussed here.

In point of fact, they are precisely the reason that we need inferential statistics. If we were somehow able to perfectly control all the conditions, then all our shots would go through the same hole every time.

 

[Don Fischer]

Want to know what your gun does average group size. I'd suggest about 10 3 or 5 shot groups then average the results. I have never been able to go out and shoot a group then turn around the same day and put the same number of holes in the same place on the next target. Simply never happens. Don't know anyone that claims he could do it either.

 

[std7mag]

All the variables you’ve mentioned are simply the sources of variance, and in no way do they invalidate the statistical methods being discussed here.

In point of fact, they are precisely the reason that we need inferential statistics. If we were somehow able to perfectly control all the conditions, then all our shots would go through the same hole every time.

Isn't that the goal??

 

[yakimanoob]

Isn't that the goal??

Not even a little. The goal is to have enough confidence in where the bullet will land so I can know whether or not to pull the trigger when I've got my reticle on an animal.

If your goal is to put as many bullets as possible through the same hole on paper, awesome. Shooting is fun, and maximizing precision is a rewarding challenge to take on.

But 1) I'm a hunter, and I'm perfectly content with some variability so long as I can be responsible in doing violence to God's creatures, and 2) putting all the bullets into one hole is actually impossible, and I like achievable goals.

 

[yakimanoob]

I found something cool! Based on the associated youtube videos, I believe they're using the Mann-Whitney U test to compare the groups, and I'm not ready to give up hope that there's a parametric test that offers better statistical precision, but it's still an awesome tool.

https://blackburndefense.com/tools/compare-groups/