The p-value ritual

The p-value and its implicit symbolism

***
The Aristotelian syllogistic logic behind the Fisherian p-value is ubiquitously misunderstood. This can lead to various fallacious logical inferences. The p-value resembles a Kantian paralogism, i.e., the metric appears objective and logically valid, even though it is not. The reliance on the p-value is an irrational social ritual (cf. Gigerenzer, 2004). Social conformity, obedience to authority, groupthink, and other aspects of Social Identity Theory (SIT) play an important role in this context.

Given the well-documented paralogisms associated with classical Fisherian null hypothesis significance testing (cf. Cohen, 1994) I advocate alternative inferential research methods. For the statistical analyses of the experimental data I collected during my PhD I utilised Bayesian bootstrapping, Bayes Factor analysis, and Bayesian parameter estimation via Markov chain Monte Carlo simulations (in addition to classical NHST).

“Few researchers are aware that their own heroes rejected what they practice routinely. Awareness of the origins of the ritual and of its rejection could cause a virulent cognitive dissonance, in addition to dissonance with editors, reviewers, and dear colleagues. Suppression of conflicts and contradicting information is in the very nature of this social ritual.”
(Gerd Gigerenzer, 2004, p. 592; Director Emeritus of the Center for Adaptive Behavior and Cognition at the Max Planck Institute for Human Development, inter alia)

Statistical Research Methods workshop at the University of Plymouth in 2014:
The pitfalls of hypothesis testing


***
Fullscreen: https://christopher-germann.de/papers/pitfalls-of-nhst/

handout-nhst
See also: https://christopher-germann.de/the-paralogisms-of-null-hypothesis-significance-testing-in-science/
Related Posts
Structural Equation Modeling (SEM) – Separating Verbal and Spatial Cognitive Faculties
Structural Equation Modeling (SEM) – Separating Verbal and Spatial Cognitive Faculties
Evaluation of statistical null-hypotheses in a Bayesian framework: A ROPE & HDI-based decision algorithm
Evaluation of statistical null-hypotheses in a Bayesian framework: A ROPE & HDI-based decision algorithm
Science Meta-Search Skript
Cognitive biases codex
Decentralised network systems of trust: Blockchain technology for scientific research
Decentralised network systems of trust: Blockchain technology for scientific research
CogNovo Forum (SMF v.2.0.14)
CogNovo Forum (SMF v.2.0.14)
BitTorrent peer-to-peer network protocol
BitTorrent peer-to-peer network protocol
α-correction for simultaneous statistical inference: Familywise error rate vs. per-family error rate
α-correction for simultaneous statistical inference: Familywise error rate vs. per-family error rate
NHST: Online Survey
NHST: Online Survey
Cognitive Liberty
Cognitive Liberty
previous arrow
next arrow
Structural Equation Modeling (SEM) – Separating Verbal and Spatial Cognitive Faculties
Structural Equation Modeling (SEM) – Separating Verbal and Spatial Cognitive Faculties
Evaluation of statistical null-hypotheses in a Bayesian framework: A ROPE & HDI-based decision algorithm
Evaluation of statistical null-hypotheses in a Bayesian framework: A ROPE & HDI-based decision algorithm
Science Meta-Search Skript
Science Meta-Search Skript
Cognitive biases codex
Cognitive biases codex
Decentralised network systems of trust: Blockchain technology for scientific research
Decentralised network systems of trust: Blockchain technology for scientific research
CogNovo Forum (SMF v.2.0.14)
CogNovo Forum (SMF v.2.0.14)
BitTorrent peer-to-peer network protocol
BitTorrent peer-to-peer network protocol
α-correction for simultaneous statistical inference: Familywise error rate vs. per-family error rate
α-correction for simultaneous statistical inference: Familywise error rate vs. per-family error rate
NHST: Online Survey
NHST: Online Survey
Cognitive Liberty
Cognitive Liberty
Evaluation of statistical null-hypotheses in a Bayesian framework: A ROPE & HDI-based decision algorithm

/*!
* Base CSS for pdf2htmlEX
* Copyright 2012,2013 Lu Wang
* https://github.com/coolwanglu/pdf2htmlEX/blob/master/share/LICENSE
*/#sidebar{position:absolute;top:0;left:0;bottom:0;width:250px;padding:0;margin:0;overflow:auto}#page-container{position:absolute;top:0;left:0;margin:0;padding:0;border:0}@media screen{#sidebar.opened+#page-container{left:250px}#page-container{bottom:0;right:0;overflow:auto}.loading-indicator{display:none}.loading-indicator.active{display:block;position:absolute;width:64px;height:64px;top:50%;left:50%;margin-top:-32px;margin-left:-32px}.loading-indicator img{position:absolute;top:0;left:0;bottom:0;right:0}}@media print{@page{margin:0}html{margin:0}body{margin:0;-webkit-print-color-adjust:exact}#sidebar{display:none}#page-container{width:auto;height:auto;overflow:visible;background-color:transparent}.d{display:none}}.pf{position:relative;background-color:white;overflow:hidden;margin:0;border:0}.pc{position:absolute;border:0;padding:0;margin:0;top:0;left:0;width:100%;height:100%;overflow:hidden;display:block;transform-origin:0 (...)

Evaluation of statistical null-hypotheses in a Bayesian framework: A ROPE & HDI-based decision algorithm
Science Meta-Search Skript

Science Search Engine

body {
margin: 0;
font-family: 'Arial', sans-serif;
overflow: hidden;
background: url('https://christopher-germann.de/system-3699542_1920.jpg') no-repeat center center fixed;
background-size: cover;
}

.header {
background-color: #333;
padding: 20px;
text-align: center;
color: #fff;
}

.container {
display: flex;
flex-direction: column;
align-items: center;
margin-top: 50px;
}

.search-form {
display: flex;
flex-direction: column;
align-items: center;
}

.search-input,
.search-select,
.search-button {
border: none;
padding: 10px;
font-size: 16px;
margin: 5px;
border-radius: 10px;
width: 300px;
box-shadow: (...)

Cognitive biases codex

# Install and load required packages
install.packages("ape")
library(ape)

# Define the cognitive biases
biases <- c("Fundamental_attribution_error",
"Implicit_bias",
"Priming_bias",
"Confirmation_bias",
"Affinity_bias",
"Self-serving_bias",
"Belief_bias",
"Framing",
"Hindsight_bias",
"Embodied_cognition",
"Anchoring_bias",
"Status_quo_bias",
"Overconfidence_effect",
"Physical_attractiveness_stereotype",
"Availability_heuristic",
"Authority_bias",
"Bystander_effect",
"Choice_supportive_bias",
"Clustering_illusion",
"Conservatism_bias",
(...)

Decentralised network systems of trust: Blockchain technology for scientific research

An interesting and innovative proposal is to use blockchain technologies (usually associated with digital crypto currencies like, for instance, Bitcoin or Ethereum) to counteract the replication crisis, to validate empirical findings, and to improve and optimize the scientific procedure on a large scale (Bartling & Fecher, 2016). The authors suggest that “Blockchain could strengthen science's verification process, helping to make more research results reproducible, true, and useful” (Bartling & Fecher, 2016, p. 1). Even though this proposal might seem unrealistic or overstated to those unfamiliar with blockchain technologies, we think that this is indeed an excellent innovative and creative proposal because blockchain technologies can be used in all situations which require (...)

Decentralised network systems of trust: Blockchain technology for scientific research
BitTorrent peer-to-peer network protocol

µTorrent - a BitTorrent client for Windows

BitTorrent is a communication protocol for peer-to-peer file sharing which is used to distribute data and electronic files over the Internet. BitTorrent is one of the most common protocols for transferring large files, such as digital video files containing TV shows or video clips or digital audio files containing songs. Peer-to-peer networks have been estimated to collectively account for approximately 43% to 70% of all Internet traffic (depending on location) as of February 2009.

To send or receive files, a person uses a BitTorrent client on their Internet-connected computer. A BitTorrent client is a computer program that implements the BitTorrent protocol. Popular clients include μTorrent, Xunlei, Transmission, qBittorrent, Vuze, Deluge, (...)

BitTorrent peer-to-peer network protocol
α-correction for simultaneous statistical inference: Familywise error rate vs. per-family error rate

/*!
* Base CSS for pdf2htmlEX
* Copyright 2012,2013 Lu Wang
* https://github.com/coolwanglu/pdf2htmlEX/blob/master/share/LICENSE
*/#sidebar{position:absolute;top:0;left:0;bottom:0;width:250px;padding:0;margin:0;overflow:auto}#page-container{position:absolute;top:0;left:0;margin:0;padding:0;border:0}@media screen{#sidebar.opened+#page-container{left:250px}#page-container{bottom:0;right:0;overflow:auto}.loading-indicator{display:none}.loading-indicator.active{display:block;position:absolute;width:64px;height:64px;top:50%;left:50%;margin-top:-32px;margin-left:-32px}.loading-indicator img{position:absolute;top:0;left:0;bottom:0;right:0}}@media print{@page{margin:0}html{margin:0}body{margin:0;-webkit-print-color-adjust:exact}#sidebar{display:none}#page-container{width:auto;height:auto;overflow:visible;background-color:transparent}.d{display:none}}.pf{position:relative;background-color:white;overflow:hidden;margin:0;border:0}.pc{position:absolute;border:0;padding:0;margin:0;top:0;left:0;width:100%;height:100%;overflow:hidden;display:block;transform-origin:0 (...)

α-correction for simultaneous statistical inference: Familywise error rate vs. per-family error rate
NHST: Online Survey

 

Suppose you have a treatment which you suspect may alter performance on a certain task. You compare the means of your control and experimental group (say 20 subjects in each sample). Further, suppose you use a simple independent means t test and your result is (t = 2.7, d.f. 18, p = 0.01).
Please mark each of the statements below as "True" or "False".
Adapted from Oakes, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. New York: Wiley.

(...)

NHST: Online Survey
previous arrow
next arrow
Shadow

Leave a Reply