Exception that proves the rule
The "exception that proves the rule", or similar variants or this phrase, is an old adage that dates from a long way before the internet laws it most resembles. It originates from the Latin phrase exceptio probat regulam.
You are most likely to encounter this phrase when somebody is speaking in generalisations or stereotypes and somebody else points out an example that clearly contradicts their comments. Retorting with the platitude that this is just the "exception that proves the rule" is an easy way of handwaving away this inconsistency.
There are some logical problems with this. While most kinds of rules have exceptions, and vague generalisations have more than most, the "exception that proves the rule" rhetoric glosses over why these exceptions exist. Any logical appraisal of a rule should regard it as weak, or even disproved, if cases can be found where it does not apply, unless there is a clear reason why it should not be applied in these cases. Instead these exceptions are said to prove the rule, suggesting paradoxically that the truth of a rule is somehow strengthened by not always being true.
 Wait... why is this even an aphorism?
Well, there's actually a bit of linguistic debate about that. There are two positions; the one from the original Latin phrase, and one from the Old English meaning of the word "prove".
 The Latin interpretation
The original meaning of the phrase was that an exception to a rule demonstrates that there is a rule in order for it to be an exception. The full Latin phrase was exceptio probat regulam in casibus non exceptis, meaning "the exception proves the rule to be true in cases not excepted". For example, on a shop sign saying "we open every day except Christmas", the exception ("except Christmas") reinforces the rule ("we open every day") to be true in all cases other than that one stated exception. Another example would be a statement that implies an unspoken rule. For instance, "free parking on Sunday" obviously implies that one must pay to park on every other day of the week. This is a rule that is proved by its exception.
 The Old-English interpretation
The phrase could also mean "the exception tests the rule", since the original meaning of "to prove" in old English (originating from the Latin word probat, compare probe) was closer to "to test". This original use of "to prove" meaning "to test" can also be seen in the phrase "proof spirit" - which was liquor which had been tested and shown to be of appropriate quality. This usage still exists in the military - a site designated for testing equipment or weaponry may be called a "proving ground" (most notably in the US Military).
In some cases, either of these definitions of "prove" create a working sentence, which somewhat explains the change in definition. For example I can say that the porn found on a priest's hard drive "proves" that he is a paedophile, and it works for either definition; it either (a) provides hard evidence of paedophilia, thus proving the accusation to be true in the modern sense of the word, or (b) provides a legitimate test of whether or not the priest is a paedophile -- if the subjects of the porn are under age, he is defined as a paedophile. However, the aphorism "the exception that proves the rule" was not treated properly by the shift in definition, and now it looks like a way that idiots can justify their idiocy.
Hence the phrase can be used correctly in the context of the scientific method, when testing a hypothesis by examining possible exceptions to it and whether they invalidate the hypothesis. So if I hypothesise that "all swans are white", the discovery of the black swan Cygnus atratus "proves" (meaning "tests") my rule. In this case, the rule is falsified.