Occam's Razor

From Teach And Discover Wiki

'From Duns Scotus' Ordinatio - 'Pluralitas non est ponenda sine necessitate - plurality is not to be posited without necessity'

Occam's razor (sometimes spelled Ockham's razor) is a principle attributed to the 14th-century English logician and Franciscan friar William of Ockham. The principle states that the explanation of any phenomenon should make as few assumptions as possible, eliminating those that make no difference in the observable predictions of the explanatory hypothesis or theory. The principle is often expressed in Latin as the lex parsimoniae ("law of parsimony" or "law of succinctness"): "entia non sunt multiplicanda praeter necessitatem", or "entities should not be multiplied beyond necessity".

This is often paraphrased as "All other things being equal, the simplest solution is the best." In other words, when multiple competing theories are equal in other respects, the principle recommends selecting the theory that introduces the fewest assumptions and postulates the fewest entities. It is in this sense that Occam's razor is usually understood.

Originally a tenet of the reductionist philosophy of nominalism, it is more often taken today as a heuristic maxim (rule of thumb) that advises economy, parsimony, or simplicity, often or especially in scientific theories.

Applications

Science and the scientific method

The aforementioned problem of underdetermination poses a serious obstacle to applications of the scientific method. Formulating theories and selecting the most promising ones is impossible without a way of choosing among an arbitrarily large number of theories, all of which fit with the evidence equally well. If any one principle could single-handedly reduce all these infinite possibilities to find the one best theory, at first glance one might deduce that the whole of scientific method simply follows from it, and thus that it alone would be sufficient to power the whole process of hypothesis formulation and rejection scientists undertake.

However, while the necessity of some method or another to determine a working hypothesis in spite of the problem of underdetermination is by and large undisputed, the progression of actual science and actual scientific consensus is far removed from some simple formula which accepts "the evidence" and outputs "the best theory". Axioms may be taken for granted that are not at all true; theories might exist that are better supported by the evidence but will be overlooked because scientists were collecting data from the wrong places or asking the wrong questions to begin with (this was emphasized by Thomas Kuhn, who outright rejected induction as the main driving force of scientific progress altogether in favor of paradigm shifts). Resorting to the importance of Occam's Razor within the limits of inductive arguments still leaves open problems of formulation; "the simplest explanation tends to be the best" is hardly a formally precise statement and it may be difficult to use it, as is, to rigorously compare two competing hypotheses. This leaves open the possibility of rigorous modern formulations, and indeed such formulations have been derived which- while being outside the scope of Occam's original razor- are true to its spirit and yield useful results (see below, "probability theory"). As a matter of fact, the razor's first known appearance, in Maimonides "The Guide for the Perplexed" was indeed done in the context of choosing between two competing scientific (cosmological) theories.

In physics, for example, one measurement of the simplicity of a theory is the number of free parameters. A theory with adjustable free parameters is considered to be less desirable than one with fewer free parameters, and a desirable goal of physics is to provide a theory with the minimum number of parameters required to explain the observations.

Occam's razor is not equivalent to the idea that "perfection is simplicity". Albert Einstein probably had this in mind when he wrote in 1933 that "The supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience" often paraphrased as "Theories should be as simple as possible, but no simpler." Or even put more simply "make it simple, not simpler". It often happens that the best explanation is much more complicated than the simplest possible explanation because its postulations amount to less of an improbability. Thus the popular rephrasing of the razor - that "the simplest explanation is the best one" - fails to capture the gist of the reason behind it, in that it conflates a rigorous notion of simplicity and ease of human comprehension. The two are obviously correlated, but hardly equivalent.

There are two senses in which Occam's razor can be seen at work in the history of science. One is ontological reduction by elimination and the other is by intertheoretic competition.

In the former case the following are examples of reduction by elimination: The impetus of Aristotelian Physics, the angelic motors of medieval celestial mechanics, the four humors of ancient and medieval medicine, demonic possession as an explanation of mental illness, phlogiston theory from premodern chemistry, and vital spirits of premodern biology.

In the latter case there are three examples from the history of science where the simpler of two competing theories each of which explains all the observed phenomena has been chosen over its ontologically bloated competitor: the Copernican heliocentric model of celestial mechanics over the Ptolemaic geocentric model, the mechanical theory of heat over the Caloric theory, and the Einsteinian theory of electromagnetism over the luminiferous aether theory.

• In the first example, the Copernican model is said to have been chosen over the Ptolemaic due to its greater simplicity. The Ptolemaic model, in order to explain the apparent retrograde motion of Mercury relative to Venus, posited the existence of epicycles within the orbit of Mercury. The Copernican model (as expanded by Kepler) was able to account for this motion by displacing the Earth from the center of the solar system and replacing it with the sun as the orbital focus of planetary motions while simultaneously replacing the circular orbits of the Ptolemaic model with elliptical ones. In addition the Copernican model excluded any mention of the crystalline spheres that the planets were thought to be embedded in according the Ptolemaic model. In a single stroke the Copernican model reduced by a factor of two the ontology of Astronomy.
• According to the Caloric theory of heat, heat is a weightless substance that can travel from one object to another. This theory arose from the study of cannon boring and the invention of the steam engine. It was while studying cannon boring that Count Rumford made observations that conflicted with the Caloric theory and he formulated his mechanical theory to replace it. The Mechanical theory eliminated the Caloric and was ontologically simpler than its predecessor.
• During the 19th century, physicists believed that light required a medium of transmission much as sound waves do. It was hypothesized that a universal aether was such a medium and much effort was expended to detect it. In one of the most famous negative experiments in the history of science, the Michelson-Morley experiment failed to find any evidence of its existence. Then when Einstein constructed his theory of special relativity without any reference to the Aether this subsequently became the accepted view, thus providing another example of a theory chosen in part for its greater ontological simplicity.

Religion

In the philosophy of religion, Occam's razor is sometimes applied to the existence of God; if the concept of God does not help to explain the universe, it is argued, God is irrelevant and should be cut away (Schmitt 2005). It is argued to imply that, in the absence of compelling reasons to believe in God, disbelief should be preferred. Such arguments are based on the assertion that belief in God requires more and more complex assumptions to explain the universe than non-belief.

The history of theistic thought has produced many arguments attempting to show that this is not the case- that the difficulties encountered by a theory without God are equal to or greater than those encountered by a theory postulating one. The cosmological argument, for example, states that the universe must be the result of a "first cause" and that that first cause must be God. Similarly, the teleological argument credits the appearance of design and order in the universe to supernatural intelligence. Many people believe in miracles or have what they call religious experiences, and creationists consider divine design to be more believable than naturalistic explanations for the diversity and history of life on earth.

The majority of the scientific community generally does not accept these arguments, and prefers to rely on explanations that deal with the same phenomena within the confines of existing scientific models. Among leading scientists defined as members of the National Academy of Sciences, 72.2% expressed disbelief and 93% expressed disbelief or doubt in the existence of a personal god in a survey conducted in 1998<ref>Larson and Witham, 1998 "Leading Scientists Still Reject God"</ref> (an ongoing survey being conducted by Elaine Ecklund of Rice University since 2004 indicates that this figure drops to as low as 38% when non-eminent scientists and social scientists are included and the definition of "God" is expanded to allow a non-personal god as per Pantheism or Deism).<ref>Ref to survey at Livescience article from Physorg.com</ref>The typical scientific view challenges the validity of the teleological argument by the effects of emergence, leading to the creation-evolution controversy; likewise, religious experiences have naturalistic explanations in the psychology of religion. Other theistic arguments, such as the argument from miracles, are sometimes pejoratively said to be arguing for a mere God of the gaps - whether or not God actually works miracles, any explanation that "God did it" must fit the facts and make accurate predictions better than more parsimonious guesses like "something did it", or else Occam's razor still cuts God out.

Rather than argue for the necessity of God, some theists consider their belief to be based on grounds independent of, or prior to, reason, making Occam's razor irrelevant. This was the stance of Søren Kierkegaard, who viewed belief in God as a leap of faith which sometimes directly opposed reason (McDonald 2005); this is also the same basic view of Clarkian Presuppositional apologetics, with the exception that Clark never thought the leap of faith was contrary to reason. (See also: Fideism). In a different vein, Alvin Plantinga and others have argued for reformed epistemology, the view that God's existence can properly be assumed as part of a Christian's epistemological structure. (See also: Basic beliefs). Yet another school of thought, Van Tillian Presuppositional apologetics, claims that God's existence is the transcendentally necessary prior condition to the intelligibility of all human experience and thought. In other words, proponents of this view hold that there is no other viable option to ultimately explain any fact of human experience or knowledge, let alone a simpler one.

Considering that the razor is often wielded as an argument against theism, it is somewhat ironic that Ockham himself was a theist. He considered some Christian sources to be valid sources of factual data, equal to both logic and sense perception. He wrote, "No plurality should be assumed unless it can be proved (a) by reason, or (b) by experience, or (c) by some infallible authority"; referring in the last clause "to the Bible, the Saints and certain pronouncements of the Church" (Hoffmann 1997). In Ockham's view, an explanation which does not harmonize with reason, experience or the aforementioned sources cannot be considered valid.

Subjective Razor

The Turing machine can be thought of as embodying a Bayesian prior belief over the space of rival theories. Hence Occam's razor is not an objective comparison method, and merely reflects the subject's prior beliefs. One's choice of exactly which razor to use is culturally relative.

Objective Razor

The minimum instruction set of a Universal Turing machine requires approximately the same length description across different formulations, and is small compared to the Kolmogorov complexity of most practical theories. For instance John Tromp's minimal universal interpreters, based on the Lambda Calculus and Combinatory logic are 210 and 272 bits respectively. Marcus Hutter has used this consistency to define a "natural" Turing machine<ref>Algorithmic Information Theory</ref> of small size as the proper basis for excluding arbitrarily complex instruction sets in the formulation of razors.

One possible conclusion from mixing these concepts - Kolmogorov complexity and Occam's Razor - is that an ideal data compressor would also be a scientific explanation/formulation generator. Some attempts have been made to re-derive known laws from considerations of simplicity or compressibility.<ref>[http://arxiv.org/pdf/math-ph/0009007 'Occam’s Razor as a formal basis for a physical theory' by Andrei N. Soklakov]</ref><ref>'Why Occam's Razor' by Russell Standish</ref>

Variations

The principle is most often expressed as Entia non sunt multiplicanda praeter necessitatem, or "Entities should not be multiplied beyond necessity", but this sentence was written by later authors and is not found in Ockham's surviving writings. This also applies to non est ponenda pluritas sine necessitate, which translates literally into English as "pluralities ought not be posited without necessity". It has inspired numerous expressions including "parsimony of postulates", the "principle of simplicity", the "KISS principle" (Keep It Simple, Stupid), and in some medical schools, "When you hear hoofbeats, think horses, not zebras".

Other common restatements are:

and

Leonardo da Vinci (1452 - 1519) lived after Ockham's time and has a variant of Occam's razor. His variant short-circuits the need for sophistication by equating it to simplicity:

Isaac Newton's version may be better: