Bonferroni's principle
| Part of a convergent series on Mathematics |
|
| 1+1=11 |
| We are number one |
Bonferroni's principle is an informal presentation of a statistical theorem that states if your method of finding significant items returns significantly more items that you would expect in the actual population, you can assume most of the items you find with it are bogus.[1] This essentially means that an algorithm or method we think is useful for finding a particular set of data actually returns more false positives as it returns larger portions of the data than should be within that category.
Informal example[edit]
Assume you are trying to identify people who are cheating on their spouses within a certain population, and you know that the percentage in the population who cheat on their spouses is 5%. If you decide that people who claim to go out with coworkers more than three times a month are most likely actually cheating on their spouses, but discover that 20% of people in the population qualify with your method, then you know in the very best case only one quarter of the people you identify will actually be cheaters. Furthermore, if there are any false negatives (cheaters who aren't identified as cheaters), an even higher percentage of the "cheaters" identified with the system would be false positives.
Uses[edit]
Applying Bonferroni's Principle to an algorithm or system for identifying or classifying data gives you an upper bound on the accuracy of your methods. If you determine that you match significantly more data or less data than you should expect than you in the best case have too many false positives or false negatives, respectively. This is not to say that the algorithm is correct in the case that it matches a number relatively close to what you would expect. You could happen to perfectly match the correct number of items in the data set but be matching the incorrect items. This is why it gives an estimation of the best case scenario.
The principle is especially useful in debunking individuals who use cold reading techniques. You may think they are using some sort of psychic power to accurately identify a single person, but if it turns out that 90% of the audience can identify with something they are saying and it's likely there's at least one person who can identify with most of what they are saying in every audience, their powers become much less impressive.
Formal Bonferroni correction[edit]
Scientific experiments that must test many hypotheses simultaneously might take Bonferroni's principle into account by raising the strength of evidence required for a positive finding. They do this by setting an overall alpha level, or maximum probability of a false positive, and testing each hypothesis at a level equal to that alpha divided by the number of hypotheses to test, called the adjusted alpha level.[2] The probability of a false positive for any individual conclusion will be no greater than the adjusted alpha, and so the probability of any false positives will be no greater than the overall alpha level.
See also[edit]
- p-hacking, when scientists intentionally ignore this principle to have a positive finding.
External links[edit]
References[edit]
- ↑ Rajaraman A, Leskovec J, Ullman J.Mining of Massive Datasets Version 1.3
- ↑ Bonferroni Correction
