In statistical hypothesis testing, there are various notions of so-called type IIIerrors, and sometimes type IV errors or higher, by analogy with the type I and type II errors of Jerzy Neyman and Egon Pearson. Fundamentally, Type III errors occur when researchers provide the right answer to the wrong question. Since the paired notions of type I errors and type II errors that were introduced by Neyman and Pearson are now widely used, their choice of terminology, has led others to suppose that certain sorts of mistakes that they have identified might be an "error of the third kind", "fourth kind", etc. None of these proposed categories has been widely accepted. The following is a brief account of some of these proposals.
Systems theory
In systems theory an additional type III error is often defined: type III : asking the wrong question and using the wrong null hypothesis.
David
a sometime colleague of both Neyman and Pearson at the University College London, making a humorous aside at the end of her 1947 paper, suggested that, in the case of her own research, perhaps Neyman and Pearson's "two sources of error" could be extended to a third:
Mosteller
In 1948, Frederick Mosteller argued that a "third kind of error" was required to describe circumstances he had observed, namely:
Type I error: "rejecting the null hypothesis when it is true".
Type II error: "accepting the null hypothesis when it is false".
Type III error: "correctly rejecting the null hypothesis for the wrong reason".
Kaiser
According to Henry F. Kaiser, in his 1966 paper extended Mosteller's classification such that an error of the third kind entailed an incorrect decision of direction following a rejected two-tailed test of hypothesis. In his discussion, Kaiser also speaks of α errors, β errors, and γ errors for type I, type II and type III errors respectively.
Kimball
In 1957, Allyn W. Kimball, a statistician with the Oak Ridge National Laboratory, proposed a different kind of error to stand beside "the first and second types of error in the theory of testing hypotheses". Kimball defined this new "error of the third kind" as being "the error committed by giving the right answer to the wrong problem". Mathematician Richard Hamming expressed his view that "It is better to solve the right problem the wrong way than to solve the wrong problem the right way". Harvard economist Howard Raiffa describes an occasion when he, too, "fell into the trap of working on the wrong problem".
Mitroff and Featheringham
In 1974, Ian Mitroff and Tom Featheringham extended Kimball's category, arguing that "one of the most important determinants of a problem's solution is how that problem has been represented or formulated in the first place". They defined type III errors as either "the error... of having solved the wrong problem... when one should have solved the right problem" or "the error... choosing the wrong problem representation... when one should have... chosen the right problem representation", p. 383. In the 2009 book Dirty rotten strategies by Ian I. Mitroff and Abraham Silvers described type III and type IV errors providing many examples of both developing good answers to the wrong questions and deliberately selecting the wrong questions for intensive and skilled investigation. Most of the examples have nothing to do with statistics, many being problems of public policy or business decisions.
Raiffa
In 1969, the Harvard economist Howard Raiffa jokingly suggested "a candidate for the error of the fourth kind: solving the right problem too late".
In 1970, L. A. Marascuilo and J. R. Levin proposed a "fourth kind of error" – a "type IV error" – which they defined in a Mosteller-like manner as being the mistake of "the incorrect interpretation of a correctly rejected hypothesis"; which, they suggested, was the equivalent of "a physician's correct diagnosis of an ailment followed by the prescription of a wrong medicine".
