In the Proceedings of the National Academy of Sciences, Johnson writes that stricter statistical tests should be required before a result is accepted. Classic statistical tests that lead to P-values are popular with scientists, but there`s also another approach known as Bayesian statistics, he explains. Latin rigorous-, stringens, present participates in stringere He says that what this study adds to the debate is the theoretical marriage of Bayesian and classical approaches. Johnson believes the results should only be considered significant at P values of 0.005 for «strong evidence» or 0.001 for «very strong evidence.» Smaller P-values lead to results that are more likely to be replicated (Source: jaroszpilewski/iStockphoto) What the P-value actually means is that there is a 5% chance that results as extreme as these will occur if there really is no difference in experience – a drug, for example, has no effect. Johnson married some special Bayesian statistical tests to classical tests, so that a direct link can be established between P values and the probability that the null hypothesis is false. The discovery could explain the growing number of studies that cannot be replicated because, according to Professor Valen Johnson of Texas A&M University, such studies may not have found a real result at all. «The threshold of 0.05 was set arbitrarily by Fischer, Neyman and Pearson in the 1930s,» Johnson explains. «They were the leading statisticians of their time, so everyone accepted it.» «This study suggests a dramatic raising of the bar,» says Professor Ian Marschner of Macquarie University in Sydney, who was not involved in the work. «I don`t think this study proves that this is the way to go, but I certainly think it`s a useful contribution and it`s a reasonable discussion for the scientific community.» «Based on that, we`re pretty sure to have 20-25 percent of the studies that aren`t reproducible because there was no real effect.» The criterion used for the recent discovery of the Higgs boson was P = 0.0000003, he adds. Joshua Stamper`s ©theme music 2006 New Jerusalem Music/ASCAP Traditionally, scientists test an alternative hypothesis against the «null hypothesis» – which represents the status quo or lack of effect. He adds that the problem is the worst in the social and life sciences.
A. Ramachandra Rao (Ph.D., Indian Institute of Statistics) is a Professor in the Department of Theoretical Statistics and Mathematics, Indian Institute of Statistics, Calcutta. He was a visiting assistant professor at the University of Minnesota for a year. He has published more than 25 research papers in graph theory, is co-author of a book on linear algebra at the honor level, and has edited the proceedings of three conferences on graph theory. His main interests are graph theory and its applications in the social sciences and linear algebra. The Bayesian approach really compares the null hypothesis to an alternative hypothesis and creates a Bayesian factor – which must be high to favor the alternative hypothesis. «When a physicist makes a discovery, it is normal for the results to be reproduced in another laboratory. They found that the use of a P-value of 0.05 was far too low to accept a new result as true. «When a result is obtained in an experiment, it is decided to accept or reject the null hypothesis. The null hypothesis is rejected and an alternative is accepted only if the probability of obtaining such an extreme result is less than a certain value assuming that the null hypothesis is true.
That`s already the case in the physics community, he says. By analyzing data from more than 800 experiments in two psychological journals, he calculated both P values and Bayesian factors for the results. «This can have a major effect in drug studies, for example. They find an effect in a phase 2 trial at P =0.05, but it`s not there when they do larger clinical trials. «The key idea of this research is that I defined Bayesian tests that reject the null hypothesis exactly as conventional tests do.» «In psychology circles, there have been efforts to explain the high number of studies that cannot be replicated,» johnson says. «They even accused scientists of inventing data. But the problem is not that the threshold for determining significance is too low. Bikas K Sinha (Ph.D., Statistics, University of Calcutta) is Professor of Statistics in the Department of Theoretical Statistics and Mathematics, Indian Institute of Statistics, Kolkata. .