User:LArron/PNAS Response to Letter 2

From RationalWiki
Jump to navigation Jump to search

PNAS Response to Letter[edit]

This part should include all comments after the first archiving - until Sep, 24th.

== Data from several experiments ==Aschlafly16:55, 20 September 2008archive
KennyMac08:20, 18 September 2008→Data from several experiments: new section
I would like to contribute to this discussion because I have taught statistics to graduate biology students for 16 years.KennyMac08:20, 18 September 2008→Data from several experiments: new section
The combination of data from several experiments is a specialist and sometimes difficult area of statistical theory but a simple example shows why Aschafly’s concern about combining the results of three different experiments is not justified and why this aspect of his criticism of Lenski’s recent paper in PNAS is not valid.KennyMac11:30, 18 September 2008→Data from several experiments
KennyMac08:20, 18 September 2008→Data from several experiments: new section
Suppose we want to conduct a test of whether or not men are taller than women on average. For the sake of the example, I generated random heights of people from a population in which men had an average height of 175cm (5’10’) and women of 165cm (5’6”). The standard deviations of height in both sexes were 7cm. I think these numbers are approximately correct for people in the UK but the details aren’t important.KennyMac08:20, 18 September 2008→Data from several experiments: new section
Suppose we take 5 samples of 2 men and 2 women. Here are the numbers I generated:KennyMac08:20, 18 September 2008→Data from several experiments: new section
{| class="wikitable"Toffeeman15:18, 20 September 2008→Data from several experiments
Toffeeman15:18, 20 September 2008→Data from several experiments
KennyMac08:20, 18 September 2008→Data from several experiments: new section
KennyMac08:20, 18 September 2008→Data from several experiments: new section
! Weighting !! Z transformed !! p Toffeeman15:18, 20 September 2008→Data from several experiments
| Equal || 4.025 || <0.001Toffeeman15:18, 20 September 2008→Data from several experiments
|By total Cit+|| 3.576 || <0.001Toffeeman15:18, 20 September 2008→Data from several experiments
|By total replicates|| 1.825 || 0.034Toffeeman15:18, 20 September 2008→Data from several experiments
So weighting on the basis of the number of replicates considerably increases the P-value. It remains, however, well within the range of statistical significance (0>P<0.05). If we hold that Lenski should have weighted on the basis of replicates then he should have rejected the null hypothesis and reached exactly the same conclusions that he did. The entire paper would have been exactly the same except the sentence “the result is extremely significant (P<0.0001) whether or not….” Would read “the result is significant (P<0.04) whether or not”. Point 5 establishes one number and an “extremely”. Point 5, therefore, has no weight (excuse the pun). Aschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman09:36, 24 September 2008Pointed out by certain gentlemen on a certain site
Toffeeman15:18, 20 September 2008→Data from several experiments
--Toffeeman 15:18, 20 September 2008 (EDT)Toffeeman15:18, 20 September 2008→Data from several experiments
: Sorry, Toffeeman, a falsehood is still a falsehood. Based on your own posting, if Lenski had applied the Whitlock Z-transform paper in a logical manner, the results would not have been nearly as striking as Lenski claimed (his paper said the results were "extremely significant"). Moreover, I found Lenski's description of his application of the Whitlock paper to be particularly misleading. Lenski's use of "whether or not"<ref>"We also used the Z-transformation method (49) to combine the probabilities from our three experiments, and the result is extremely significant (P < 0.0001) whether or not the experiments are weighted by the number of independent Cit+ mutants observed in each one." (Lenski paper at 7902).</ref> obscures the basic error that he did not apply Whitlock's paper in the straightforward, correct manner. I think the wording in the Lenski paper deliberately obscures this falsehood from the reader.Aschlafly16:47, 20 September 2008reply re: Point 5
: People have free will to embrace and defend falsehoods. I don't expect them to change quickly or admit they were wrong. But you'll find me defending and promoting the truth.Aschlafly16:47, 20 September 2008reply re: Point 5
: Point 5 remains valid and the falsehood remains uncorrected by PNAS or Lenski. Four other points in higher priority remain uncorrected by them also.--Aschlafly 16:47, 20 September 2008 (EDT)Aschlafly16:47, 20 September 2008reply re: Point 5
::"a falsehood is still a falsehood". Precisely, the null hypothesis should be considered false and the conclusions of the paper stand.Toffeeman17:13, 20 September 2008→Point 5 Confirmed
::Oh? Do you mean Lenski's falsehood? And what falsehood is that? Lenski said that he had calculated the P-value without weighting and that had come out at <0.0001. That is true, not false. Lenski said he had calculated the P-value weighting on the basis of the Cit+ replicates and that had come out at <0.0001. That is true, not false. There is no falsehood. Lenski did not mention the results of weighting on the basis of replicate numbers. He thus made no claim about weighting on the basis of replicate numbers. If he made no claim he cannot have made a false claim. Toffeeman17:13, 20 September 2008→Point 5 Confirmed
::How do the words Lenski uses "mislead". If you read them as written what conclusion do you come to? You are lead to the conclusion that the mutation was not "rare-but-equal", instead it was contingent. That is the right conclusion. If Lenski had presented the data in a different manner (perhaps by including the results of weighting on the basis of replicate numbers) what conclusion do you come to? You are again lead to the conclusion that the mutation was not "rare-but-equal", instead it was contingent. That is the right conclusion. To mislead you must be lead to a conclusion that is incorrect. By Lenski's paper you are not lead to a conclusion that is incorrect. Thus is cannot be said to be "misleading". Toffeeman17:13, 20 September 2008→Point 5 Confirmed
::I shall not comment on your second paragraph, the temptation to "Tu Quoque" would be too great. Toffeeman17:14, 20 September 2008→Point 5 Confirmed
Toffeeman17:13, 20 September 2008→Point 5 Confirmed
--Toffeeman 17:13, 20 September 2008 (EDT)Toffeeman17:13, 20 September 2008→Point 5 Confirmed
::: The falsehood consists of pretending to apply the Whitlock Z-transform in a straightforward, logical and correct manner. I think the Lenski paper is intentionally misleading by using the "whether or not" wording, when both alternatives are nonsensical. Point 5 has been proven above to be correct in identify an error in the Lenski paper.Aschlafly17:54, 20 September 2008→Point 5 Confirmed: reply
::: "Toffeeman", your blocking history suggests you have been less than straightforward yourself. Go elsewhere if you seek to be deceitful. You're not fooling anyone here.--Aschlafly 17:54, 20 September 2008 (EDT)Aschlafly17:54, 20 September 2008→Point 5 Confirmed: reply
::::Lenski did apply the Z-transformation correctly, both weighted and unweighted. The data points extracted from each replay are the generation numbers of those replicates that gave rise to Cit+ mutants. Thus Replay 1 produced four data points: 30,500 31,500 32,500 32,500. Replay 2 produced five data points: 32,000 32,000 32,000 32,000 32,500. Replay 3 produced eight data points: 20,000 20,000 27,000 27,000 31,000 31,500 32,000 32,000. Thus the N for replay 1 is 4; replay 2 is 5 and replay 3 is 8. The fact that replay 3 used 38 times as many replicates as replay 1 does not mean that it should be weighted 38 times as much; it only produced twice as much data, not 38 times as much. Brossa00:06, 21 September 2008→Point 5 Confirmed
::::Suppose I want to find out what the average age of a murderer is in three cities. In L.A. I interview 72 random people and find that 4 of them were convicted of murder; I record the ages of the four. In Seattle I interview 340 people and find that 5 of them are convicted murderers; likewise in Singapore I interview 2800 and find 8 murderers. In the end, I have 4,5, and 8 data points from the three cities; the number of people I had to interview to obtain those data points doesn't factor into the analysis of what the average age of the murderers is.--Brossa 00:06, 21 September 2008 (EDT)Brossa00:06, 21 September 2008→Point 5 Confirmed
::::: In your first paragraph you simply repeat the error underlying Lenski's paper. You, like the paper, incorrectly apply Whitlock's Z-transform.Aschlafly11:30, 21 September 2008repy to Brossa
::::: The quality and reliability of data is proportional to sample size, and when different studies are combined they need to be weighted accordingly. The results from a very large sample size would not be weighted equally with the results from a small sample size, as you and Lenski have done. That's basic logic, though I'm not optimistic that you or Lenski will admit it. Open-minded people who respect logic have no difficulty elevating logic over personal whim.-Aschlafly 11:30, 21 September 2008 (EDT)Aschlafly11:30, 21 September 2008repy to Brossa
:::::: So, what is the p-value you find when performing the calculations according to your way? --Stitch75 12:03, 21 September 2008 (EDT)Aschlafly12:13, 21 September 2008Reverted edits by Stitch75 (Talk); changed back to last version by Aschlafly
Stitch7512:03, 21 September 2008→Point 5 Confirmed
::::::Andy if you read Whitlock's paper you would see it say, and I quote, "Ideally each study is weighted proportional to the inverse of its error variance, that is, by the reciprocal of its squared standard error." It says nothing about weighting according to sample size, which is what you seem to be insisting should be done.DanB20:40, 21 September 2008→Point 5 Confirmed
DanB20:39, 21 September 2008→Point 5 Confirmed
::::::Also Whitlock acknowledges in the paper that there is no preference for weighted versus equal weighting, so the fact that both equal weighting and weighting by the standard error give a statistically significant result shows that the 3 experiments combined support rejection of the null hypothesis. DanB 20:39, 21 September 2008 (EDT)DanB20:39, 21 September 2008→Point 5 Confirmed
:::::::ASchlafly, you state that I incorrectly apply "Whitlock's Z-transform" (actually the test belongs to Mosteller & Bush<ref>Mosteller, F. & Bush, R.R. 1954. Selected quantitative techniques. In: Handbook of Social Psychology, Vol. 1 (G. Lindzey, ed.)</ref> and/or Liptak<ref>Liptak, T. 1958. On the combination of tests. Magyar Tud. Akad. Mat. Kutato Int. Kozl. 3: 171-197</ref>). Whitlock describes weighting by the reciprocal of the squared standard error. The standard error of the mean is proportional to 1/sqrt(N), so the reciprocal of the squared standard error is proportional to N. Thus larger studies are given more weight. I maintain that the sample sizes N of the three replays are 4, 5, and 8 respectively. Weighting based on those three N does not weight all three replays equally as you claim: it gives replay 2 25% more weight and replay 3 100% more weight than replay 1. Rather than simply repeating that I am wrong, will you please state what you think the sample sizes of the three replay experiments are, and, in your opinion, what the correct application of the Z-transformation would be?--Brossa 18:00, 22 September 2008 (EDT) Brossa19:21, 22 September 2008response to ASchlafly and DanB
Brossa18:00, 22 September 2008()
:::::::: The Lenski paper states how it weighted the experiments, and that weighting is incorrect. Admit it. Moreover, the incorrect weighting in the Lenski paper was not likely an inadvertent error, as it inflated the significance of the results. I found the wording used by the Lenski paper to describe its (incorrect) weighting to be artfully misleading.Aschlafly08:35, 23 September 2008reply Brossa
:::::::: Provide me with federal funding as Lenski received, and I'll write a paper for you. But I don't have to write an alternative paper to point out glaring errors in Lenski's paper.--Aschlafly 08:35, 23 September 2008 (EDT)Aschlafly08:35, 23 September 2008reply Brossa
::::::::: I've been following this discussion for a while and I have to agree with ASchlafly. It hardly seems fair that he should have to, in his spare time, replicate an experiment done by a professional just to "earn" the right to criticize it. I am unfamiliar with statistics, but if some complicated transform goes against common sense, common sense should prevail. After all, there are lies, damned lies, and statistics... AndyM 10:57, 23 September 2008 (EDT)AndyM10:57, 23 September 2008()
(unindent)I'm not asking anyone to write a paper or replicate an experiment. I'm asking ASchlafly to support his statement "The results from a very large sample size would not be weighted equally with the results from a small sample size, as you and Lenski have done"(bolding mine). I have stated publicly, subject to challenge by others, that the sample sizes (n) of the three replays are four, five, and eight respectively. Furthermore, using n of 4, 5, and 8 in the weighted Z-method DOES NOT weight all the replay experiments equally - it weights replay 3 twice as much as replay 1 and 8/5 as much as replay 2. Tell you what: I'll drop all my questions about Monte Carlo and the Z-transform, and simply ask ASchlafly one question: what is the sample size, n, of the second replay experiment? He need not even do any calculations - a statement in words that will allow someone else to do the calculation will suffice. This is not a complicated question to answer; the paper states how many replicate cultures there were (340), how many cells there were in each replicate (3.9x10^8), how many replicates gave rise to Cit+ cells (5), and which generations those Cit+ replicates came from (4 from 32,000 and one from 32,500). I will even give my answer: five. Furthermore, I will say why I believe that, using the murderer/age analogy: performing the 340 replicates is the same as interviewing 340 people in order to find out if any of them are convicted murderers. Finding that five replicates gave rise to Cit+ mutants is the same as the survey finding that 5 of those 340 people were convicted murderers. Finding that the Cit+ mutants arose from 4 replicates from generation 32,000 and 1 from generation 32,500 is the same as finding the ages of the murderers. The five data points in the Lenski study allow one to calculate the 'mean generation of clones yielding Cit+': 32,100. This is the same as finding the mean age of the five murderers. If I want to compare this hypothetical murderer age study to some other study of the mean age of murderers, I would weight the studies based on how many murderers were in each study, not on how many non-murderers were included in the initial survey. Brossa15:54, 23 September 2008()
Surely ASchlafly can say what he thinks the n of the second replay is, even if he won't say why he thinks it. Is it five? 340? The number of replicates times the number of cells per replicate? Something else? No analysis need be performed on the resulting number.--Brossa 15:54, 23 September 2008 (EDT)Brossa15:54, 23 September 2008()
:OK - would you care to put in writing that after ASchlafly gives you his response, you won't start obfuscating the issue with Monte Carlo and Z-transform issues? You understand: it is typical of liberals to, after being proven wrong, to start pretending that they were talking about an entirely different issue altogether. After ASchlafly states the sample size of the second replay experiment you will consider yourself answered. Correct?AndyM16:57, 23 September 2008()
:Brossa, your rant is misplaced. One cannot salvage an error in logic by questioning which of superior alternatives should be used instead. The sample size of an experiment is the number that comprises the underlying sample used in the experiment, not the number of a certain outcome from the experiment. Maybe you can debate yourself over what the correct underlying sample size is, but it is plainly not the number of a certain outcome from the experiment.--Aschlafly 19:28, 23 September 2008 (EDT)Aschlafly19:28, 23 September 2008reply to Brossa
(non-substantive postings deleted; the rule for this page is enforced.--Aschlafly 17:58, 24 September 2008 (EDT))Aschlafly17:58, 24 September 2008non-substantive postings deleted
Brossa07:34, 24 September 2008()
:::Aschlafly, are we not getting back to the distinction I made here? A suggestion that you rejected?Aschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman07:37, 24 September 2008()
:::If we are agreed on what constitutes the sample then it is clear that Lenski did combine the sample in a logical manner with proper weighting. He followed accepted practice and used established statistical techniques. Your letter did not dispute the soundness of accepted practice and established techniques, it alleged that Lenski had diverted from them. If the sample is agreed then this allegation is untrue.Aschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman07:37, 24 September 2008()
:::If, contrary to your earlier statement, there is a dispute about the sample then it is Lenski who is correct on the sample size. Lenski is testing a hypothesis about Cit+ replicates. If you were testing (to use Brossa’a example) “most murders are committed by 20 to 35 year olds” you would include murderers, you would not run statistics on the age of fraudsters. You may get your data from a source that includes fraudsters, say prison records that include details of all types of criminals. However you would exclude non-murderers, and correctly exclude non-murderers, when reviewing the age of murderers. Aschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman09:35, 24 September 2008()
Toffeeman07:37, 24 September 2008()
:::That is the situation we have here. Lenski’s null hypothesis is that Cit+ replicates arose randomly. This says no more about Cit- replicates than it does about what Lenski had for lunch the day he wrote the paper. The presence of Cit- replicates is a result of the method of generating the sample, a side product. To give another example, if you were doing an experiment breeding chickens you would generate a lot of manure: you wouldn’t include it in the analysis!Aschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman09:35, 24 September 2008()
Toffeeman07:37, 24 September 2008()
:::Finally the argument has no end product: if we erroneously consider the Cit- replicates as part of the sample the null hypothesis is still refuted. Your objections to Lenski’s wording do not establish deceit on his part, yet they would were they to be established themselves. Your objections to Lenski’s calculations, however, would not establish the falsity of his conclusions even if the objections were correct. The calculations of the three differing weightings above are not intended:Aschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman09:35, 24 September 2008()
Toffeeman07:37, 24 September 2008()
::::1. to show that Lenski was honest (he was, but that is established by other evidence) Aschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman07:37, 24 September 2008()
::::2. to show that Lenski correctly applied accepted statistical techniques (he did, but that is established by other evidence) orAschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman07:37, 24 September 2008()
::::3. to show that only Cit+ replicates are part of “the sample” (they are, but that is established elsewhere)Aschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman07:37, 24 September 2008()
:::The calculations of the three differing weightings above show that your objections are moot. --Toffeeman 07:37, 24 September 2008 (EDT)Aschlafly17:56, 24 September 2008Reverted edits by Toffeeman (Talk); changed back to last version by Brossa
Toffeeman07:37, 24 September 2008()
:After reading the deleted postings in the history I'm not sure you did the right thing ASchlafly. They should stay here, or be moved to the liberal style page that they perfectly illustrate. On the other hand, deleting them keeps readers from having to sort through that trash: three different weighting styles?? Surely Brossa and Toffeeman can at least attempt an honest argument without having to conjure up new mathematics. AndyM 19:15, 24 September 2008 (EDT)AndyM19:15, 24 September 2008()
Maybe the Journal of Nature can be your next letter submission source. International Weekly Journal of Science -- 14px jp 21:17, 22 September 2008 (EDT)Aschlafly19:57, 24 September 2008moved non-substantive comments to "random talk"
Aschlafly19:57, 24 September 2008moved non-substantive comments to "random talk"
Jpatt21:17, 22 September 2008()
== Random Talk ==Aschlafly19:57, 24 September 2008moved non-substantive comments to "random talk"
 Aschlafly11:31, 23 September 2008Reverted edits by Jirby (Talk); changed back to last version by AndyM
Jirby11:27, 23 September 2008→Point 5 Confirmed
Aschlafly16:56, 20 September 2008→Lack of statistical expertise: removed rant in violation of rule
TomG16:53, 20 September 2008→Data from several experiments
Sorry, I really can't let this slide. Andy, by claiming that you've taken and excelled in upperclass statistics courses you are in effect claiming expertise in the subject. Anyone with experitse, or even basic knowledge of statistics, would not have written Mystery:Young Hollywood Breast Cancer Victims, wherein you argue the following: There are currently 500 "Hollywood" women under 30 who would disclose a diagnosis or breast cancer (a made up number, but we don't need to dwell on that). We have an acknowledgement of 3 "Hollywood" under 30 women ever who have been diagnosed with breast cancer. Therefore the rate of breast cancer in Hollywood women under 30 is at least 3 in 500 (more than 10 times the national average). If you can't figure out the blatant error in that statement then there is no way on earth you can expect any of us to believe that you are capable of finding flaws in others statistical analysis. While it is possible that you did once "excel" in high level statistics courses, it is pretty obvious you have not retained what you learned. Nothing wrong with that, but you shouldn't pretend to have expertise you don't have. TomG 16:53, 20 September 2008 (EDT)Aschlafly16:56, 20 September 2008→Lack of statistical expertise: removed rant in violation of rule
TomG16:53, 20 September 2008→Data from several experiments
== References ==Aschlafly16:47, 20 September 2008added reference
<references/>Aschlafly16:47, 20 September 2008added reference
! Expt# !! p-value !! z-score !! #Cit+ muts !! #replicatesToffeeman15:18, 20 September 2008→Data from several experiments
| 1 || 0.0085 || 2.387 || 4 || 72Toffeeman15:18, 20 September 2008→Data from several experiments
| 2 || 0.0007 || 3.195 || 5 || 340Toffeeman15:18, 20 September 2008→Data from several experiments
| 3 || 0.0823 || 1.390 || 8 || 2800Toffeeman15:18, 20 September 2008→Data from several experiments
Applying the formula described above.. Toffeeman15:18, 20 September 2008→Data from several experiments
! Men mean !! Women mean !! Mean difference !! t !! PKennyMac11:30, 18 September 2008→Data from several experiments
KennyMac08:20, 18 September 2008→Data from several experiments: new section
| 175.6 || 164.6 || 11 || 3.85 || 0.00058KennyMac08:20, 18 September 2008→Data from several experiments: new section
Clearly, combining the data from several similar experiments strengthens the conclusions considerably, as shown by the fact that P is much smaller for the combined data than for any individual sample.KennyMac11:30, 18 September 2008→Data from several experiments
KennyMac08:20, 18 September 2008→Data from several experiments: new section
Although the combination of data from several experiments is a specialised area of statistics, I see nothing particularly incorrect about the approach used by Lenski and his colleagues. The general point is that it is valid to combine the results of different experiments if it is scientifically meaningful to do so. (For example: A. Combining the results of five samples of the heights of men and women is clearly valid. B. Combining three samples of heights of men and women with two samples of lengths of male and female squid clearly isn’t.) Generally speaking, the outcome of a combined analysis of several small experiments which all point in the same direction (or at least in a similar direction) will be more significant than that of any one of those experiments, as is shown in the larger table above.KennyMac11:45, 18 September 2008→Data from several experiments: A little clarification
KennyMac08:20, 18 September 2008→Data from several experiments: new section
I hope this clarifies the extensive discussion on this point and puts Aschafly’s mind at rest on this subject. KennyMac 08:20, 18 September 2008 (EDT)KennyMac08:20, 18 September 2008→Data from several experiments: new section
:That's very nicely put, thanks. You should work on some of the stats pages here. Of course, technically any sample is ultimately just a combination of n samples of size 1. MikeR 13:28, 18 September 2008 (EDT)MikeR13:28, 18 September 2008→Data from several experiments
:I'll take a look at this Friday. It's not immediately obvious what the point is to your analysis above.--Aschlafly 23:46, 18 September 2008 (EDT)Aschlafly23:46, 18 September 2008→Data from several experiments: not immediately obvious
::"The general point is that it is valid to combine the results of different experiments if it is scientifically meaningful to do so"--KingOfNothing 00:57, 19 September 2008 (EDT)KingOfNothing00:57, 19 September 2008→Data from several experiments
:::This makes no sense as an argument. It may be true in this simple case that you can do one large or several small samples and get similar results - which is quite obvious and wouldn't need such a detailed rant. However, you provide no mathematical proof, just one example. Etc 01:08, 19 September 2008 (EDT)Etc01:08, 19 September 2008→Data from several experiments
::KingOfNothing is correct. The point is that, if a series of experiments all indicate (or tend to indicate) the same conclusion, the result of analysing the complete series of tests may be much stronger than that of any one test (i.e. it may have a much lower significance test probability). This is as true of Lenski's experiments as of the simple example above. Hence Aschafly's criticism of Lenski's paper in this respect is invalid.DeanS10:11, 19 September 2008Undo revision 520261 by KennyMac (Talk) Sock of Blocked users
KennyMac09:59, 19 September 2008→Data from several experiments: Answering Aschafly's query
:: Regarding Etc's point: If you paraphrase Aschafly's argument in his Letter to PNAS as the hypothesis that one weak result should cast doubt on several stronger results, even though they all tend to lead to the same conclusion, the example above disproves that hypothesis. No further mathematical analysis is necessary. (By the way, a carefully argued text is not a rant.) KennyMac 09:59, 19 September 2008 (EDT)DeanS10:11, 19 September 2008Undo revision 520261 by KennyMac (Talk) Sock of Blocked users
KennyMac09:59, 19 September 2008→Data from several experiments: Answering Aschafly's query
:"It's not immediately obvious what the point is to your analysis above". No surprise, Aschlafly, really. Maybe you should take your own advice: "I suggest you try harder with an open mind". --CrossC 02:46, 19 September 2008 (EDT)CrossC02:46, 19 September 2008→Data from several experiments: sig
CrossC02:45, 19 September 2008→Data from several experiments
It is with great sadness that I note that the author of this - the only significant statistical explanation and discussion in this entire fiasco- has just been blocked for five years. Even his email is blocked, so he can't even appeal the action. I don't see such manouvers as having contributed to the much vaunted "open mind" of which various people here speak. BenHur 10:27, 19 September 2008 (EDT)BenHur10:27, 19 September 2008→Data from several experiments
REPLY: I have now reviewed the above analysis, and it supports Point 5 rather than the PNAS paper. Point 5 stated, "The Third Experiment was erroneously combined with the other two experiments based on outcome rather than sample size, thereby yielding a false claim of overall statistical significance." The analysis above does nothing more than reinforce Point 5 by combining experiments based on sample size.Aschlafly19:24, 19 September 2008reply
TomG10:30, 19 September 2008→Data from several experiments
In Pavlovian manner, some Lenski types nod their head here in agreement at the above analysis, apparently unaware that it reinforces Point 5.Aschlafly19:24, 19 September 2008reply
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
When combining results from samples that are vastly different in sample size, it is necessary to factor in the different sample sizes. Apparently the PNAS paper failed to do that, which helps explain why it refuses to provide a meaningful response to Point 5.--Aschlafly 19:24, 19 September 2008 (EDT)Aschlafly19:24, 19 September 2008reply
(rants below were deleted for being non-substantive in violation of this page's rules.)--Aschlafly 19:24, 19 September 2008 (EDT)Aschlafly19:24, 19 September 2008reply
AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
1.Figure 3 depicts an "historical contingency" hypothesis around the 31,000th generation, but the abstract states that mutations "arose by 20,000 generations." The paper fails to admit that the Third Experiment disproved the contingency hypothesis depicted in Figure 3. Aschlafly19:24, 19 September 2008reply
AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
Reading over the paper, I didn’t find anything about a “historical contingency” around the 31,000th generation. What I believe I am seeing is a misunderstanding, Lenski was looking for two scenarios, First he indicates that if the ability for the E. Coli to consume citrate was due to a mutation, than the rate of the mutation would remain constant, but if it was population that made the mutation more likely, there would be a point where the mutation rate would increase. The graph simply showed both options, so I believe there was a misunderstanding of the graph. Aschlafly19:24, 19 September 2008reply
AlexaJ10:35, 19 September 2008Respond this time, please.
:::I understand point 5 now, or at least I think I do. What we have as a "sample" is either:Toffeeman19:57, 19 September 2008()
::::1. Individual cultures (Schlafly)Toffeeman19:57, 19 September 2008()
::::2. Cultures that developed cit+. (Lenski)Toffeeman19:57, 19 September 2008()
:::Schlafly contends that the sample should be all the cultures and that Lenski has, improperly, filtered the sample by excluding the vast majority of it (i.e. all those cultures that did not become cit+). Am I right in thinking this is the argument? --Toffeeman 19:57, 19 September 2008 (EDT)Toffeeman19:57, 19 September 2008()
:::: No, we're talking about how Lenski combined a large study (which did not really support Lenski's hypothesis) with small studies (which Lenski claims does support his hypothesis). The studies were not combined in a logical manner with proper weighting given to the much bigger size of the large study.--Aschlafly 23:15, 19 September 2008 (EDT)Aschlafly23:15, 19 September 2008reply to Toffeeman
:::::Aschlafly, have you read the paper on z-transforms which explains the statistical technique used? You can download a .pdf copy of the paper for free here. --BillA 06:30, 20 September 2008 (EDT)BillA06:30, 20 September 2008→Data from several experiments
:::::: You say "statistical technique used," but you should have said "statistical technique cited." In fact, a close reading of the Z-transform paper provides more support for Point 5: combined studies must be weighted based on sample size: Aschlafly09:34, 20 September 2008reply re: Point 5
::::::: "When there is variation in the sample size across studies, there can be a noticeable difference in the power of the two methods, with the weighted Z-approach being superior in all cases. As such, we should always prefer the weighted Z to the unweighted Z-approach when the independent studies test the same hypothesis."see p. 1371.Aschlafly09:34, 20 September 2008reply re: Point 5
:::::: In other words, the cited paper actually supports Point 5.--Aschlafly 09:34, 20 September 2008 (EDT)Aschlafly09:34, 20 September 2008reply re: Point 5
(unindent)Lenski used the weighted method. See note 49 to the paper and the text around the combination. Of course there is the question of on what basis Lenski weighted the results. Lenski weighted the results on the basis of the Cit+ numbers and we may think it would have been better to weight on the basis of the number of replicates. I have below the calculations (not mine) of combined P-values based on 1) no weighting, 2) weighting on the basis of Cit+ and 3) weighting on the basis of replicates. The weighted Z-transformed = SUM(Weight x Z-score for each run)/SQRT(SUM(Weight^2 for each run))Toffeeman15:19, 20 September 2008→Data from several experiments
Toffeeman15:18, 20 September 2008→Data from several experiments
Toffeeman14:33, 20 September 2008→Data from several experiments
2.Both hypotheses propose fixed mutation rates, but the failure of mutations to increase with sample size disproves this. If the authors claim that it is inappropriate to compare for scale the Second and Third Experiments to each other and to the First Experiment, then it was also an error to treat them similarly statistically. Aschlafly19:24, 19 September 2008reply
AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
2.Both hypotheses propose fixed mutation rates, but the failure of mutations to increase with sample size disproves this. If the authors claim that it is inappropriate to compare for scale the Second and Third Experiments to each other and to the First Experiment, then it was also an error to treat them similarly statistically.AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
I am not sure what you are trying to point out here. As the PNAS pointed out, mutations are random. Another note is that Lenski adressed this in his paper, and highlighted it in the article, even pointing out changes in the third experiment. Just a question, what do you mean by treating “them similarly statistically.”?AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
3. The paper incorrectly applied a Monte Carlo resampling test to exclude the null hypothesis for rarely occurring events. The Third Experiment results are consistent with the null hypothesis, contrary to the paper's claim. Aschlafly19:24, 19 September 2008reply
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
Erm… why was the Monte Carlo resampling incorrect? I cannot make heads nor tail of what you were thinking when you wrote this. Also, the result was inconsistant with the null hypothesis by a margin of 92 percent.AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
4. It was error to include generations of the E. coli already known to contain trace Cit+ variants. The highly improbable occurrence of four Cit+ variants from the 32,000th generation in the Second Experiment suggests an origin from undetected, pre-existing Cit+ variants. Aschlafly19:24, 19 September 2008reply
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
First off, modern technology grants quite a decent ability to detect the amount of bacteria in the sample, and there were NO Cit+ variants in it, nor were they mentioned in the paper. Honestly, do you believe that Lenski and his assistants were playing “toss the bacteria around”? Your assumption that there *had* to be Cit+ bacteria in the sample is incorrect, and seems to be based off your misunderstanding of the field. Aschlafly19:24, 19 September 2008reply
AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
5. The Third Experiment was erroneously combined with the other two experiments based on outcome rather than sample size, thereby yielding a false claim of overall statistical significance. Aschlafly19:24, 19 September 2008reply
AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
Again… what? There is simply nothing “wrong” or “incorrect” about combining different sized samples to test the results. There are correct techniques for testing this kind of things (“Z-transformation” method, to be specific!) and it was used to make the test. Aschlafly19:24, 19 September 2008reply
AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
Finally, your claim about the paper not being publicly available is false, as you can easily find it by going to Lenski’s site All the information is right there, 3 clicks away from a Google search. Aschlafly19:24, 19 September 2008reply
AlexaJ11:12, 19 September 2008→Dear Aschlafly,: The link was somehow removed...
AlexaJ10:35, 19 September 2008Respond this time, please.
Might I ask you to clarify the questions and insight I have provided above? AlexaJAschlafly19:24, 19 September 2008reply
AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
Might I ask you to clairify the questions and insight I have provided above?AlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
Waiting for your response, AlexaJAlexaJ10:35, 19 September 2008Respond this time, please.
AlexaJ22:42, 18 September 2008→Dear Aschlafly,: new section
: Your posting is filled with gross misspellings, "I don't get it" type of statements, and a general link that shows nothing. I suggest you try harder with an open mind. Thanks and Godspeed.--Aschlafly 23:47, 18 September 2008 (EDT)Aschlafly19:24, 19 September 2008reply
Aschlafly23:47, 18 September 2008→Dear Aschlafly,: reply ... try harder
::... 7 slightly misspelled words doesn't give an apt enough excuse not to respond, Andy.Jirby 01:03, 19 September 2008 (EDT)Aschlafly19:24, 19 September 2008reply
Jirby01:03, 19 September 2008→Dear Aschlafly,
::Can we assume from your lack of response to these points (and others above) that you don't have the expertise to answer these questions? It's okay if you don't, you should just be humble and admit it. JohnDee 09:37, 19 September 2008 (EDT)Aschlafly19:24, 19 September 2008reply
JohnDee09:37, 19 September 2008→Dear Aschlafly,
:::Are you not making an assumption here, JohnDee? If you were in his position, would you have answered a letter with all these mistakes and a condescending tone?--Carolyn 09:54, 19 September 2008 (EDT)Aschlafly19:24, 19 September 2008reply
Carolyn09:54, 19 September 2008→Dear Aschlafly,
::::Read over this talk page. Andy has never directly answered any criticism of his PNAS letter. He either removes the comment from the talk page, bans the user, or just refuses to answer the question. Sometimes all three. Given this and the criticism I've seen on this page from people who seem to know what they are talking about, the only conclusion I can come to is Andy doesn't actually understand why z-transformations are applied, what a Monte Carlo re-sampling test is, etc. If he understood any of this, I would assume he would actually answer their questions. JohnDee 10:20, 19 September 2008 (EDT)Aschlafly19:24, 19 September 2008reply
JohnDee10:23, 19 September 2008→Dear Aschlafly,
JohnDee10:20, 19 September 2008→Dear Aschlafly,
::::"Are you not making an assumption here". Spelling mistakes signify little beyond spelling. If you were to consider swimming to the bottom of the Pacific and I said "you won't be able to, you can't breathe underwater" then I would be right. If I were to write "U wont bee able to, yu can't breev undewatta" I would still be right about your ability to swim to the bottom of the Pacific. Woefully wrong about spelling, but spot on about the status of your gills. On the "tone" of the edit I might remind you that PNAS took exception to the "tone" of the letter. "Mr. Schlafly's final comment about release of data is uncalled for". Professor Lenski certainly took exception to the tone of both emails to him: "(g)iven the continued rudeness of your second email" (my emphasis). And, naturally, it makes little difference to the validity of the criticisms: if I were to say "you *******************, you can't breathe underwater" I would still be right. So "your spelling is bad" or "you are being rude" simply fail to negate the statement "the letter was wrong". In the abscence of a negation coupled with plenty of corroboration, we a forced to the conclusion that the letter had little merit. --Toffeeman 10:32, 19 September 2008 (EDT)Aschlafly19:24, 19 September 2008reply
Toffeeman10:32, 19 September 2008()
:::::I am resting under a similar assumption. My next couple of questions, Aschlafly, first off, why did you refuse to give a direct answer to a single question, instead option to say "Your spelling was bad, hence, you are closed minded." I think you mean to say "I have no idea how to give a correct response, hence I will just refuse to answer you." Until you actually answer my questions, I will assume you do not understand how to answer. Second, yes, there are "I don't understand" parts in my questions, parts I wanted you to clarify so I could understand them. Refusing to answer those makes it look, again, like you simply don't understand, in this case, you don't understand what YOU wrote. Finally, the "general link" was a link to the cumulative research of Richard Lenski, so that point is completely invalid. If you could respond to my post WITHOUT brushing it off, it would be deeply appreciated. (by the way, I re-checked the spelling just for you.) AlexaJAschlafly19:24, 19 September 2008reply
AlexaJ10:35, 19 September 2008→Dear Aschlafly,: Sig and spacing.
AlexaJ10:35, 19 September 2008Respond this time, please.
:::::::Just for our own awareness, can I please ask you AlexaJ what your expertise in this subject matter is? Thanks--Carolyn 10:45, 19 September 2008 (EDT)DeanS10:50, 19 September 2008Reverted edits by Carolyn (Talk); changed back to last version by AlexaJ
Carolyn10:45, 19 September 2008→Dear Aschlafly,
Schlafly, admitting you were wrong does not in any way reflect weakness, nor does it have to affect your belief in any way. On the contrary, admitting you are wrong proves you are sure of your capacities to better yourself, and that you are willing to learn. If you say it right, you will also have a good chance to eanr the respect of some of the people that criticized you. DeanS16:10, 19 September 2008Reverted edits by Fred1776 (Talk); changed back to last version by AlexaJ
Fred177615:36, 19 September 2008→Dear Aschlafly,
Continuous denial, and refusal to answer because of a few spelling mistakes, on the other hand reflects fear of making mistakes, and has the effect of worsening your image. You choose. Fred1776 15:36, 19 September 2008 (EDT)DeanS16:10, 19 September 2008Reverted edits by Fred1776 (Talk); changed back to last version by AlexaJ
Fred177615:36, 19 September 2008→Dear Aschlafly,
! Man1 !! Man2 !! Woman1 !! Woman2 !! Men mean !! Women mean !! Mean difference !! t !! PKennyMac08:20, 18 September 2008→Data from several experiments: new section
| 176 || 179 || 157 || 148 || 177.5 || 152.5 || 25 || 5.27 || 0.017KennyMac08:20, 18 September 2008→Data from several experiments: new section
| 180 || 176 || 160 || 164 || 178 || 162 || 16 || 5.66 || 0.015KennyMac08:20, 18 September 2008→Data from several experiments: new section
| 176 || 175 || 167 || 165 || 175.5 || 166 || 9.5 || 8.50 || 0.0068KennyMac08:20, 18 September 2008→Data from several experiments: new section
| 169 || 171 || 168 || 173 || 170 || 170.5 || -0.5 || -0.19 || 0.57KennyMac08:20, 18 September 2008→Data from several experiments: new section
| 179 || 175 || 166 || 178 || 177 || 172 || 5 || 0.79 || 0.26KennyMac08:20, 18 September 2008→Data from several experiments: new section
P in the last column is the t-test probability for a one-side test of women being shorter than men. (Formally, it’s the probability of getting a value of t greater than that calculated from the data if women are in fact taller than men on average.)KennyMac08:20, 18 September 2008→Data from several experiments: new section
Should the fact that, in the fourth sample, the average height of the women is taller than that of the men make us doubt that men are in fact taller on average? Should we be concerned about the last sample, in which the difference in height of the two sexes is rather small, though in the expected direction? No, in both cases. When we combine the data on all 10 men and all 10 women, we get this:KennyMac11:45, 18 September 2008→Data from several experiments: A little clarification
KennyMac08:20, 18 September 2008→Data from several experiments: new section
== Please ==Bugler14:06, 13 September 2008()
NewUser12:52, 13 September 2008→Please: new section
Please, stop deleting entries in this talk page. It puts conservatives in bad light. Let's answer with reason and debating skills instead. This is particularly aimed at user:Aschlafly.Bugler14:06, 13 September 2008()
NewUser12:52, 13 September 2008→Please: new section
Notice: misrepresentations are not going to be allowed on this page. Substantive comments only, please.Aschlafly10:13, 13 September 2008reply
Note: earlier posts are archived here. --BillA 17:05, 20 September 2008 (EDT)BillA17:05, 20 September 2008()
Retrieved from "https://rationalwiki.org/w/index.php?title=User:LArron/PNAS_Response_to_Letter_2&oldid=940867"