The study of scientific discovery - where do new ideas come from? - has long been denigrated by philosophers as irrelevant to analyzing the growth of scientific knowledge. In particular, little is known about how cognitive theories are discovered, and neither the classical accounts of discovery as either probabilistic induction (e.g., Reichenbach, 1938) or lucky guesses (e.g., Popper, 1959) nor the stock anecdotes about sudden "eureka" moments deepen understanding of discovery.
The tools-to-theories heuristic describes how scientists' tools shape theories of mind, in particular with how methods of statistical inference have been transformed into metaphors of mind, and explains the emergence of a broad range of cognitive theories, from the cognitive revolution of the 1960s up to the present. It can be used to detect both limitations and new lines of development in current cognitive theories that investigate the mind as an "intuitive statistician."
Although psychology’s most important task is to integrate the various extant patchworks of theories into overarching theories, theory integration remains a poor relative in the field. As a consequence, in some parts of psychology, theories have become replaced by surrogates, such as tautological descriptions of the phenomenon, one-word explanations, and lists of general dichotomies. Moving backwards from existing models to labels is an irregular occurrence in science, which typically progresses in the opposite direction.
The “replication crisis” has been attributed to misguided external incentives gamed by researchers (the strategic-game hypothesis). At the same time, there is a complementary internal factor, namely, researchers’ widespread faith in a statistical ritual and associated delusions. The “null ritual,” unknown in statistics proper, eliminates judgment precisely at points where statistical theories demand it. The crucial delusion is that the p value specifies the probability of a successful replication (i.e., 1 – p), which makes replication studies appear to be superfluous. Psychology departments need to begin teaching statistical thinking, not rituals, and journal editors should no longer accept manuscripts that report results as “significant” or “not significant.”