The problem calls into question all empirical claims made in everyday life or through the scientific method. Although the problem arguably dates back to the Pyrrhonism of ancient philosophy, David Hume introduced it in the mid-18th century, with the most notable response provided by Karl Popper two centuries later. A more recent, probability-based extension is the "no-free-lunch theorem for supervised learning" of Wolpert and Macready.
In inductive reasoning, one makes a series of observations and infers a new claim based on them. For instance, from a series of observations that at sea-level (approximately 14.7 psi, or 101 kPa) water freezes at 0°C (32°F), it seems valid to infer that the next sample of water will do the same, or that, in general, at sea-level water freezes at 0°C. That the next sample of water freezes under those conditions merely adds to the series of observations. First, it is not certain, regardless of the number of observations, that water always freezes at 0°C at sea-level. To be certain, it must be known that the law of nature is immutable. Second, the observations themselves do not establish the validity of inductive reasoning, except inductively.
Pyrrhonian skeptic Sextus Empiricus first questioned the validity of inductive reasoning, positing that a universal rule could not be established from an incomplete set of particular instances. He wrote:
When they propose to establish the universal from the particulars by means of induction, they will effect this by a review of either all or some of the particulars. But if they review some, the induction will be insecure, since some of the particulars omitted in the induction may contravene the universal; while if they are to review all, they will be toiling at the impossible, since the particulars are infinite and indefinite.
The focus upon the gap between the premises and conclusion present in the above passage appears different from Hume's focus upon the circular reasoning of induction. However, Weintraub claims in The Philosophical Quarterly that although Sextus' approach to the problem appears different, Hume's approach was actually an application of another argument raised by Sextus:
Those who claim for themselves to judge the truth are bound to possess a criterion of truth. This criterion, then, either is without a judge's approval or has been approved. But if it is without approval, whence comes it that it is truthworthy? For no matter of dispute is to be trusted without judging. And, if it has been approved, that which approves it, in turn, either has been approved or has not been approved, and so on ad infinitum.
Although the criterion argument applies to both deduction and induction, Weintraub believes that Sextus' argument "is precisely the strategy Hume invokes against induction: it cannot be justified, because the purported justification, being inductive, is circular." She concludes that "Hume's most important legacy is the supposition that the justification of induction is not analogous to that of deduction." She ends with a discussion of Hume's implicit sanction of the validity of deduction, which Hume describes as intuitive in a manner analogous to modern foundationalism.
Medieval writers such as al-Ghazali and William of Ockham connected the problem with God's absolute power, asking how we can be certain that the world will continue behaving as expected when God could at any moment miraculously cause the opposite. Duns Scotus however argued that inductive inference from a finite number of particulars to a universal generalization was justified by "a proposition reposing in the soul, 'Whatever occurs in a great many instances by a cause that is not free, is the natural effect of that cause.'" Some seventeenth century Jesuits argued that although God could create the end of the world at any moment, it was necessarily a rare event and hence our confidence that it would not happen very soon was largely justified.
David Hume described the problem in An Enquiry concerning Human Understanding, §4, based on his epistemological framework. Here, "reason" refers to deductive reasoning and "induction" refers to inductive reasoning.
First, Hume ponders the discovery of causal relations, which form the basis for what he refers to as "matters of fact." He argues that causal relations are found not by reason, but by induction. This is because for any cause, multiple effects are conceivable, and the actual effect cannot be determined by reasoning about the cause; instead, one must observe occurrences of the causal relation to discover that it holds. For example, when one thinks of "a billiard ball moving in a straight line toward another," one can conceive that the first ball bounces back with the second ball remaining at rest, the first ball stops and the second ball moves, or the first ball jumps over the second, etc. There is no reason to conclude any of these possibilities over the others. Only through previous observation can it be predicted, inductively, what will actually happen with the balls. In general, it is not necessary that causal relation in the future resemble causal relations in the past, as it is always conceivable otherwise; for Hume, this is because the negation of the claim does not lead to a contradiction.
Next, Hume ponders the justification of induction. If all matters of fact are based on causal relations, and all causal relations are found by induction, then induction must be shown to be valid somehow. He uses the fact that induction assumes a valid connection between the proposition "I have found that such an object has always been attended with such an effect" and the proposition "I foresee that other objects which are in appearance similar will be attended with similar effects." One connects these two propositions not by reason, but by induction. This claim is supported by the same reasoning as that for causal relations above, and by the observation that even rationally inexperienced or inferior people can infer, for example, that touching fire causes pain. Hume challenges other philosophers to come up with a (deductive) reason for the connection. If he is right, then the justification of induction can be only inductive. But this begs the question; as induction is based on an assumption of the connection, it cannot itself explain the connection.
In this way, the problem of induction is not only concerned with the uncertainty of conclusions derived by induction, but doubts the very principle through which those uncertain conclusions are derived.
Nelson Goodman presented a different description of the problem of induction in the article "The New Problem of Induction" (1966). Goodman proposed a new predicate, "grue". Something is grue if and only if it has been observed to be green before a certain time or blue after that time. The "new" problem of induction is, since all emeralds we have ever seen are both green and grue, why do we suppose that after time t we will find green but not grue emeralds? The standard scientific response is to invoke Occam's razor.
Goodman, however, points out that the predicate "grue" only appears more complex than the predicate "green" because we have defined grue in terms of blue and green. If we had always been brought up to think in terms of "grue" and "bleen" (where bleen is blue before time t, or green thereafter), we would intuitively consider "green" to be a crazy and complicated predicate. Goodman believed that which scientific hypotheses we favour depend on which predicates are "entrenched" in our language.
W.V.O. Quine offers the most practicable solution to this problem by making the metaphysical claim that only predicates that identify a "natural kind" (i.e. a real property of real things) can be legitimately used in a scientific hypothesis.
Although induction is not made by reason, Hume observes that we nonetheless perform it and improve from it. He proposes a descriptive explanation for the nature of induction in §5 of the Enquiry, titled "Skeptical solution of these doubts". It is by custom or habit that one draws the inductive connection described above, and "without the influence of custom we would be entirely ignorant of every matter of fact beyond what is immediately present to the memory and senses." The result of custom is belief, which is instinctual and much stronger than imagination alone.
Rather than unproductive radical skepticism about everything, Hume said that he was actually advocating a practical skepticism based on common sense, wherein the inevitability of induction is accepted. Someone who insists on reason for certainty might, for instance, starve to death, as they would not infer the benefits of food based on previous observations of nutrition.
Colin Howson interpreted Hume to say that an inductive inference must be backed not only by observations, but also by an independent "inductive assumption." Howson combined this idea with Frank P. Ramsey's view on probabilistic reasoning to conclude that "there is a genuine logic of induction which exhibits inductive reasoning as logically quite sound given suitable premisses, but does not justify those premisses." In this sense, the strength of inductive reasoning is comparable to that of deductive reasoning.
Karl Popper, a philosopher of science, sought to resolve the problem of induction in the context of the scientific method. He argued that science does not rely on induction, but exclusively on deduction, by making the modus tollens argument form the centerpiece of his theory. Knowledge is gradually advanced as tests are made and failures are accounted for.
Wesley C. Salmon critiques Popper's falsifiability by arguing that in using corroborated theories, induction is being used. Salmon stated, "Modus tollens without corroboration is empty; modus tollens with corroboration is induction."
The problem of induction exists within the philosophy of science, but not within the philosophy of mathematics, for "induction" refers to different concepts in the two fields. Mathematical induction is indeed a form of deductive reasoning.