Bayesian parameter estimation supersedes t-test


The Bayesian inferential approach provides rich information about the estimated distribution of several parameters of interest, i.e., it provides the distribution of the estimates of μ and σ of both experimental conditions and the associated effect sizes. Specifically, the method provides the “relative credibility” of all possible differences between means, standard deviations (Kruschke, 2013). Inferential conclusions about null hypotheses can be drawn based on these credibility values. In contrast to conventional NHST, uninformative (and frequently misleading124) p values are redundant in the Bayesian framework. Moreover, the Bayesian parameter estimation approach enables the researcher to accept null hypotheses. NHST, on the other, only allows the researcher to reject such null hypotheses. The critical reader might object why one would use complex Bayesian computations for the relatively simple within-group design at hand. One might argue that a more parsimonious analytic approach is preferable. Exactly this question has been articulated before in a paper entitled “Bayesian computation: a statistical revolution” which was published in the Philosophical Transactions of the Royal Society: “Thus, if your primary question of interest can be simply expressed in a form amenable to a t test, say, there really is no need to try and apply the full Bayesian machinery to so simple a problem” (S. P. Brooks, 2003, p. 2694). The answer is straightforward: “Decisions based on Bayesian parameter estimation are better founded than those based on NHST, whether the decisions derived by the two methods agree or not. The conclusion is bold but simple: Bayesian parameter estimation supersedes the NHST t test” (Kruschke, 2013, p. 573).
Bayesian parameter estimation is more informative than NHST125 (independent of the complexity of the research question under investigation). Moreover, the conclusions drawn from Bayesian parameter estimates do not necessarily converge with those based on NHST. This has been empirically demonstrated beyond doubt by several independent researchers (Kruschke, 2013; Rouder et al., 2009).

The function ‘BEST.R’ can be downloaded from the CRAN (Comprehensive R Archive Network) repository under the following URL:
https://cran.r-project.org/web/packages/BEST/index.html

The BEST function has been ported to MATLAB and Python.

Matlab version of BEST: https://github.com/NilsWinter/matlab-bayesian-estimation/
Python version of BEST: https://github.com/strawlab/best/

The posterior distribution is approximated by a powerful class of algorithms known as Markov chain Monte Carlo (MCMC) methods (named in analogy to the randomness of events observed at games in casinos). MCMC generates a large representative sample from the data which, in principle, allows to approximate the posterior distribution to an arbitrarily high degree of accuracy (as 𝑡→∞). The MCMC sample (or chain) contains a large number (i.e., > 1000) of combinations of the parameter values of interest. Our model of perceptual judgments contains the following parameters: < μ1, μ2, σ1, σ2, 𝜈 > (in all reported experiments). In other words, the MCMC algorithm randomly samples a very large n of combinations of θ from the posterior distribution. This representative sample of θ values is subsequently utilised in order to estimate various characteristics of the posterior (Gustafsson, Montelius, Starck, & Ljungberg, 2017), e.g., its mean, mode, median/medoid, standard deviation, etc. The thus obtained sample of parameter values can then be plotted in the form of a histogram in order to visualise the distributional properties and a prespecified high density interval (i.e., 95%) is then superimposed on the histogram in order to visualise the range of credible values for the parameter under investigation.

References

Kruschke, J. K.. (2013). Bayesian estimation supersedes the t test.. Journal of Experimental Psychology: General, 142(2), 573–603.

Plain numerical DOI: 10.1037/a0029146
DOI URL
directSciHub download

Kruschke, J. K.. (2013). Posterior predictive checks can and should be Bayesian: Comment on Gelman and Shalizi, ‘Philosophy and the practice of Bayesian statistics’. British Journal of Mathematical and Statistical Psychology, 66(1), 45–56.

Plain numerical DOI: 10.1111/j.2044-8317.2012.02063.x
DOI URL
directSciHub download

Kruschke, J. K.. (2011). Introduction to Special Section on Bayesian Data Analysis. Perspectives on Psychological Science, 6(3), 272–273.

Plain numerical DOI: 10.1177/1745691611406926
DOI URL
directSciHub download

Kruschke, J. K., & Meredith, M.. (2012). BEST Manual – Mike Meredith. R-CRAN

Plain numerical DOI: 10.1037/a0029146
DOI URL
directSciHub download

Older posts

[ajax_load_more]

Leave a Reply

Your email address will not be published. Required fields are marked *