ToxSci Advance Access originally published online on May 29, 2007
Toxicological Sciences 2007 98(2):599-601; doi:10.1093/toxsci/kfm135
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Limitations in the National Cancer Institute Antitumor Drug Screening Database for Evaluating Hormesis
Environ Corporation, 602 East Georgia Avenue, Ruston, LA 71270
1 To whom correspondence should be addressed. Fax: (318) 325-4889. E-mail: KennyCrump{at}email.com.
Received February 9, 2007; accepted March 5, 2007
Calabrese et al. (2006)
presented an analysis of data from the U.S. National Cancer Institute (NCI) yeast screening database that appeared to demonstrate hormesis. However, their method of analysis was biased toward finding hormesis even if none was present. Although Calabrese et al. maintained that this bias was not sufficient to explain their results, in reaching this conclusion they assumed that when the original data were summarized, the responses in exposed wells were normalized by dividing by the average of the responses in the eight control wells. Alternatively (and not implausibly), if the responses in exposed wells were first divided by responses in individual control wells and the ratios averaged, results remarkably similar to those obtained by Calabrese et al. could have occurred, even if hormesis were not present. Unfortunately, the original data were lost and there is apparently no way to determine which method was used to summarize them (Julian Simon, personal communication).
Briefly, in the NCI yeast screening study, 13 strains of yeast were exposed to five molar concentrations (1.2, 3.7, 11, 33, and 100µM) of 2189 potential anticancer drugs. The experiments were conducted using 96-well plates, with 80 wells of each plate being exposed to the same molar concentration of 80 different chemicals, and eight of the remaining wells exposed only to solvent. All the wells of each plate contained a single strain of yeast, and all strains were exposed simultaneously on 13 separate plates to the same molar concentration. After incubation, the optical density in each well was measured as the indicator of cell growth. Each plate configuration was replicated four times, and the summary data available to Calabrese et al. consisted of the average of the four replicate responses, each of which was normalized (in some manner) by the responses from the eight solvent control wells located on the same plate. Calabrese et al. estimated a NOEL (no-observed-effect-level) in each yeast strain for each chemical, and examined the responses below these NOELs for evidence of hormesis. Although Calabrese et al. expressed their results in terms of the benchmark dose or BMD, we express results equivalently in terms of the NOEL, which is the largest experimental molar concentration less than the BMD.
In one analysis Calabrese et al. (2006; Table 3) tabulated the average and standard deviation of percent responses by yeast strain (13 values) by NOEL value (four values). These 48 averages were all above 100%, ranged from 101.7% to 108.8% and with an overall average of 105.8%. Calabrese et al. interpreted these results as evidence for hormesis, based on the notion that in the absence of hormesis, these averages should be clustered around 100% response. However, this assumption is not generally valid. Even if control and treatment responses have the same mean, the mean of the ratio of the two responses will be greater than 100% (Loeve, 1963, p. 159). Calabrese et al. claimed that in their analysis this bias would be very slight and presented one example in which the expected average would be only 100.1%. However, in reaching this conclusion they assumed that the summary responses in the yeast data base were obtained by first averaging the responses from the eight solvent control wells, and then dividing the response in the exposed well by this control average. However, another plausible way to normalize the data would be to first divide the response in the exposed well by each of the control responses, and then average the eight ratios, i.e., by dividing and then averaging, rather than averaging and then dividing.
I have conducted a simulation to determine the effect of these two methods of averaging on the analyses conducted by Calabrese et al. In each simulation 100,000 sets of four replicates of responses from both an exposed well and eight solvent control wells were simulated. With each replicate in each set a single summary ratio was obtained, either by first averaging the eight controls, and then dividing the result for the exposed well by this average (i.e., averaging and then dividing), or by dividing and then averaging. The summarized response for each set was then calculated as the average of the four replicates. Both exposed and individual control values had the same log-normal distribution, which guaranteed that that there was no hormetic effect in the simulated data. The log mean of the log-normal distribution was set at zero and the log standard deviation was adjusted to give approximately the same standard deviation for the summarized responses as was obtained from the yeast data by Calabrese et al.
Table 1 compares the results of this simulation with those obtained by Calabrese et al. from the actual yeast data. With either averaging method the standard deviation was, as planned, very similar to the overall value of 13.3 obtained by Calabrese et al. from the yeast data. When controls were averaged first, the average response was 100.7% which is considerably below the value of 105.8% obtained by Calabrese et al. (although higher than the value of 100.1% predicted by Calabrese et al.). However, when ratios were taken first, the agreement between the average response from simulated data (105.6%) was remarkably close to the overall average (105.8%) obtained by Calabrese et al. Thus, if the yeast data were summarized by first taking ratios and then averaging, the results in Table 3 of Calabrese et al. can be explained without the need to invoke hormesis.
|
A second analysis by Calabrese et al. (2006
; Table 4) consisted of comparing the number of responses above certain percentages (100%, 105%, 115%, and 120%) to the number of responses below the corresponding complimentary percentages (100%, 1/1.05 = 95.2%, 1/1.1 = 90.9%, 1/1.15 = 87.0%, and 1/1.2 = 83.3%). The rationale stated by Calabrese et al. for this analysis was that if there were no effect of dose below the NOEL, the responses should vary randomly around 100%, whereas hormesis would cause an excess of responses above 100%. Although this would be true if the responses consisted simply of ratios of a single exposed value to a single control value, that simple situation does not apply here, since the responses were averages of four replicates, each of which consisted of an exposed response normalized (in some way) by eight control responses. In this more complicated situation the percentages are not necessarily clustered around 100%. To evaluate this analysis, the data from the simulation described above were summarized in the same way as used by Calabrese et al. in their Table 4. Results from this exercise are shown in Table 2. With neither summarization method did responses vary randomly around 100%, as assumed by Calabrese et al. Thus, the claim by Calabrese et al. that "the threshold model predicts a ratio closely approximating 1:1" cannot be supported for either summarization method. When controls were averaged before division, the number of results above the upper percentile was consistently below the number of results below the lower percentile. More importantly, when ratios were computed first and then averaged, the opposite occurred. In this case, the ratio obtained from the simulated data was 1.7:1, which is considerably greater than 1:1 and practically identical to the ratio (1.8:1) obtained by Calabrese et al. from the yeast data. As the ratio between the upper and lower percentage increases (i.e., as one moves progressively toward the right-hand side of Table 2) the ratio of simulated values above the upper percentile to the values below the percentile increases in a manner qualitatively similar to that seen in the yeast data, although the magnitude of the increase is less than that in the yeast data. It is possible that other distributional assumptions would bring the simulated results even more in line with those obtained Calabrese et al. Modifying the simulation so that 80 exposed responses shared the same eight control responses (to simulate the fact that each plate contained 80 exposed wells that all shared the same eight controls) made essentially no difference in these results.
|
A third analysis by Calabrese et al. (their Fig. 5) compares the predicted distribution of counts above and below 100% with the distribution expected under the assumption that a response is equally likely to be above or below 100%. However, our simulation results indicate that this assumption would generally be invalid (no matter which summarization method was used), even if there was no hormesis.
These simulation results demonstrate that, if the NCI yeast screening database was summarized by first taking ratios of responses in exposed wells and then averaging these ratios, then the results obtained by Calabrese et al. can be reasonably explained without the presence of hormesis. Since apparently there is no way to determine with any certainty which summarization method was employed, the claim by Calabrese et al. that the database demonstrates hormesis must be considered premature.
In addition to the potential positive bias discussed herein, sources of negative bias are also potentially present. There is a possibility of some residual dose-related reduction in cell growth below the estimated NOEL, which would tend to counteract any hormetic effect that might be present. In addition, as noted by Calabrese et al., growth in peripheral wells can be slightly less than in interior wells, due to slightly greater evaporation in the peripheral well (Faessel et al., 1999). This could cause a negative bias in the yeast data because all of the controls were placed on the periphery of the plate. The existence and amount of such bias is likely to depend on experimental details that are unique to each study. The possibility of such effects was not studied in the yeast screening study (Dr Julian Simon, personal communication).
Despite the possibility of these sources of negative bias, we see no way to reliably evaluate their potential, or even confirming their presence, in the yeast database. Thus, they provide no basis for altering our position that the analysis by Calabrese et al. cannot be used to substantiate a claim of hormesis in the NCI antitumor yeast screening database. In fact, missing both the original data and the documentation on exactly how the data were summarized into their present form, it is difficult to see how anything definitively can be said about the presence or absence of hormesis in these data.
ACKNOWLEDGMENTS
The cooperation of Dr Julian Simon, who was in charge of generating the yeast data, is gratefully acknowledged.
REFERENCES
Calabrese EJ, Staudenmayer JW, Stanek EJ III, Hoffmann GR. Hormesis outperforms threshold model in National Cancer Institute antitumor drug screening database. Toxicol. Sci. (2006) 94(2):368–378.
Faessel HM, Levasseur LM, Slocum HK, Greco WR. Parabolic growth patterns in 96-well plate cell growth experiments. In Vitro Cell. Dev. Biol. (1999) 35:270–278.[CrossRef][Web of Science]
Loeve M. Probability Theory (1963) Princeton, NJ: D. Van Nostrand Company, Inc.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  K. Elliott A case for deliberation in response to hormesis research Human and Experimental Toxicology, July 1, 2008; 27(7): 529 - 538. [Abstract] [PDF]
|
|
|
|
 |
|
 O.-N. Bae, K.-M. Lim, J.-Y. Han, B.-I. Jung, J.-Y. Lee, J.-Y. Noh, S.-M. Chung, M.-Y. Lee, J.-Y. Lee, and J.-H. Chung U-shaped Dose Response in Vasomotor Tone: A Mixed Result of Heterogenic Response of Multiple Cells to Xenobiotics Toxicol. Sci., May 1, 2008; 103(1): 181 - 190. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||