In the late 19th century, "luminiferous aether" (or "ether"), meaning light-bearing aether, was the term used to describe a medium for the propagation of light. The word aether stems via Latin from the Greek αιθήρ, from a root meaning to kindle, burn, or shine. It signifies the substance which was thought in ancient times to contain the manipulative forces beyond control.
Later theories including special relativity were formulated without the concept of aether. Today the idea of aether, what Albert Michelson called "one of the grandest generalizations of modern science", is regarded as a superseded scientific theory.
Isaac Newton contended that light was made up of numerous small particles. This could explain such features as light's ability to travel in straight lines and reflect off surfaces. This theory was known to have its problems: although it explained reflection well, its explanation of refraction and diffraction was less satisfactory. In order to explain refraction, Newton's Opticks (1704) postulated an "Aethereal Medium" transmitting vibrations faster than light, by which light, when overtaken, is put into "Fits of easy Reflexion and easy Transmission", which caused refraction and diffraction. Newton believed that these vibrations were related to heat radiation:
Is not the Heat of the warm Room convey'd through the vacuum by the Vibrations of a much subtiler Medium than Air, which after the Air was drawn out remained in the Vacuum? And is not this Medium the same with that Medium by which Light is refracted and reflected, and by whose Vibrations Light communicates Heat to Bodies, and is put into Fits of easy Reflexion and easy Transmission?
The modern understanding is that heat radiation is, like light, electromagnetic radiation. However, Newton considered them to be two different phenomena. He believed heat vibrations to be excited "when a Ray of Light falls upon the Surface of any pellucid Body". He wrote, "I do not know what this Aether is", but that if it consists of particles then they must be "exceedingly smaller than those of Air, or even than those of Light: The exceeding smallness of its Particles may contribute to the greatness of the force by which those Particles may recede from one another, and thereby make that Medium exceedingly more rare and elastick than Air, and by consequence exceedingly less able to resist the motions of Projectiles, and exceedingly more able to press upon gross Bodies, by endeavoring to expand itself."
Christiaan Huygens, prior to Newton, had hypothesized that light was a wave propagating through an aether, but Newton rejected this idea. The main reason for his rejection stemmed from the fact that both men could apparently only envision light to be a longitudinal wave, like sound and other mechanical waves in fluids. However, longitudinal waves by necessity have only one form for a given propagation direction, rather than two polarizations as in a transverse wave, and thus they were unable to explain the phenomenon of birefringence, where two polarizations of light are refracted differently by a crystal. Instead, Newton preferred to imagine non-spherical particles, or "corpuscles", of light with different "sides" that give rise to birefringence. A further reason why Newton rejected light as waves in a medium was because such a medium would have to extend everywhere in space, and would thereby "disturb and retard the Motions of those great Bodies" (the planets and comets) and thus "as it [light's medium] is of no use, and hinders the Operation of Nature, and makes her languish, so there is no evidence for its Existence, and therefore it ought to be rejected."
In 1720 James Bradley carried out a series of experiments attempting to measure stellar parallax. Although he failed to detect any parallax, thereby placing a lower limit on the distance to stars, he discovered another effect, stellar aberration, an effect which depends not on position (as in parallax), but on speed. He noticed that the apparent position of the star changed as the Earth moved around its orbit. Bradley explained this effect in the context of Newton's corpuscular theory of light, by showing that the aberration angle was given by simple vector addition of the Earth's orbital velocity and the velocity of the corpuscles of light, just as vertically falling raindrops strike a moving object at an angle. Knowing the Earth's velocity and the aberration angle, this enabled him to estimate the speed of light. To explain stellar aberration in the context of an aether-based theory of light was regarded as more problematic, because it requires that the aether be stationary even as the Earth moves through it—precisely the problem that led Newton to reject a wave model in the first place.
However, a century later, Young and Fresnel revived the wave theory of light when they pointed out that light could be a transverse wave rather than a longitudinal wave—the polarization of a transverse wave (like Newton's "sides" of light) could explain birefringence, and in the wake of a series of experiments on diffraction the particle model of Newton was finally abandoned. Physicists still assumed, however, that like mechanical waves, light waves required a medium for propagation, and thus required Huygens's idea of an aether "gas" permeating all space.
However, a transverse wave apparently required the propagating medium to behave as a solid, as opposed to a gas or fluid. The idea of a solid that did not interact with other matter seemed a bit odd, and Augustin-Louis Cauchy suggested that perhaps there was some sort of "dragging", or "entrainment", but this made the aberration measurements difficult to understand. He also suggested that the absence of longitudinal waves suggested that the aether had negative compressibility. George Green pointed out that such a fluid would be unstable. George Gabriel Stokes became a champion of the entrainment interpretation, developing a model in which the aether might be (by analogy with pine pitch) rigid at very high frequencies and fluid at lower speeds. Thus the Earth could move through it fairly freely, but it would be rigid enough to support light.
Later, Maxwell's equations showed that light is an electromagnetic wave. The apparent need for a propagation medium for such Hertzian waves can be seen by the fact that they consist of perpendicular electric (E) and magnetic (B or H) waves. The E waves consist of undulating dipolar electric fields, and all such dipoles appeared to require separated and opposite electric charges. Electric charge is an inextricable property of matter, so it appeared that some form of matter was required to provide the alternating current that would seem to have to exist at any point along the propagation path of the wave. Propagation of waves in a true vacuum would imply the existence of electric fields without associated electric charge, or of electric charge without associated matter. Albeit compatible with Maxwell's equations, electromagnetic induction of electric fields could not be demonstrated in vacuum, because all methods of detecting electric fields required electrically charged matter.
In addition, Maxwell's equations required that all electromagnetic waves in vacuum propagate at a fixed speed, c. As this can only occur in one reference frame in Newtonian physics (see Galilean-Newtonian relativity), the aether was hypothesized as the absolute and unique frame of reference in which Maxwell's equations hold. That is, the aether must be "still" universally, otherwise c would vary along with any variations that might occur in its supportive medium. Maxwell himself proposed several mechanical models of aether based on wheels and gears, and George FitzGerald even constructed a working model of one of them. These models had to agree with the fact that the electromagnetic waves are transverse but never longitudinal.
Nevertheless, by this point the mechanical qualities of the aether had become more and more magical: it had to be a fluid in order to fill space, but one that was millions of times more rigid than steel in order to support the high frequencies of light waves. It also had to be massless and without viscosity, otherwise it would visibly affect the orbits of planets. Additionally it appeared it had to be completely transparent, non-dispersive, incompressible, and continuous at a very small scale. 
Maxwell wrote in Encyclopedia Britannica:
Aethers were invented for the planets to swim in, to constitute electric atmospheres and magnetic effluvia, to convey sensations from one part of our bodies to another, and so on, until all space had been filled three or four times over with aethers.... The only aether which has survived is that which was invented by Huygens to explain the propagation of light.
Contemporary scientists were aware of the problems, but aether theory was so entrenched in physical law by this point that it was simply assumed to exist. In 1908 Oliver Lodge gave a speech in behalf of Lord Rayleigh  to the Royal Institution on this topic, in which he outlined its physical properties, and then attempted to offer reasons why they were not impossible. Nevertheless he was also aware of the criticisms, and quoted Lord Salisbury as saying that "aether is little more than a nominative case of the verb to undulate". Others criticized it as an "English invention", although Rayleigh jokingly corrected them to state it was actually an invention of the Royal Institution.
By the early 20th Century, aether theory was in trouble. A series of increasingly complex experiments had been carried out in the late 1800s to try to detect the motion of earth through the aether, and had failed to do so. A range of proposed aether-dragging theories could explain the null result but these were more complex, and tended to use arbitrary-looking coefficients and physical assumptions. Lorentz and Fitzgerald offered within the framework of Lorentz ether theory a more elegant solution to how the motion of an absolute aether could be undetectable (length contraction), but if their equations were correct, the new special theory of relativity (1905) could generate the same mathematics without referring to an aether at all. Aether fell to Occam's Razor.
The key difficulty with the aether hypothesis arose from the juxtaposition of the two well-established theories of Newtonian dynamics and Maxwell's electromagnetism. Under a Galilean transformation the equations of Newtonian dynamics are invariant, whereas those of electromagnetism are not. Basically this means that while physics should remain the same in non-accelerated experiments, light would not follow the same rules because it is traveling in the universal "aether frame". Some effect caused by this difference should be detectable.
A simple example concerns the model on which aether was originally built: sound. The speed of propagation for mechanical waves, the speed of sound, is defined by the mechanical properties of the medium. For instance, if one is in an airliner, you can still carry on a conversation with the person beside you because the sound of your words are traveling along with the air inside the aircraft. This effect is basic to all Newtonian dynamics, which says that everything from sound to the trajectory of a thrown baseball should all remain the same in the aircraft as sitting still on the Earth. This is the basis of the Galilean transformation, and the concept of frame of reference.
But the same was not true for light, since Maxwell's mathematics demanded a single universal speed for the propagation of light, based, not on local conditions, but on two measured properties, the permittivity and permeability of free space, that were assumed to be the same throughout the universe. If these numbers did change, there should be noticeable effects in the sky; stars in different directions would have different colors, for instance. Certainly they would remain constant within a small volume, inside the aircraft in our example for instance, which implies that light would not follow along with the aircraft (or the Earth) in a fashion similar to sound. Nor could light change media, for instance, using the atmosphere while near the Earth. It had already been demonstrated that if this were so, the sky would be colored in different directions as the light moved from the still medium of the aether to the moving medium of the Earth's atmosphere, causing diffraction.
Thus at any point there should be one special coordinate system, "at rest relative to the aether". Maxwell noted in the late 1870s that detecting motion relative to this aether should be easy enough—light traveling along with the motion of the Earth would have a different speed than light traveling backward, as they would both be moving against the unmoving aether. Even if the aether had an overall universal flow, changes in position during the day/night cycle, or over the span of seasons, should allow the drift to be detected.
Numerous experiments were carried out in the late 1800s to test for this "aether wind" effect, but most were open to dispute due to low accuracy. Measurements on the speed of propagation were so inaccurate that comparing two speeds to look for a difference was essentially impossible.
The famous Michelson-Morley experiment instead compared the source light with itself after being sent in different directions, looking for changes in phase in a manner that could be measured with extremely high accuracy. The publication of their result in 1887, the null result, was the first clear demonstration that something was seriously wrong with the aether concept of that time. A series of experiments using similar but increasingly sophisticated apparatus all returned the null result as well. Conceptually different experiments that also attempted to detect the motion of the aether ware the 1903 Trouton-Noble experiment in 1903 and the experiments to detect double refraction by Lord Rayleigh in 1902 and DeWitt Bristol Brace in 1904, which like Michelson-Morley obtained a null result.
In this case the MM experiment yielded a shift of the fringing pattern of about 0.01 of a fringe, corresponding to a small velocity. However, it was incompatible with the expected aether wind effect due to the earth's (seasonally varying) velocity which would have required a shift of 0.4 of a fringe, and the error was small enough that the value may have indeed been zero. Therefore, the null hypothesis, the hypothesis that there was no aether wind, could not be rejected. More modern experiments have since reduced the possible value to a number very close to zero, about 10−15.
These "aether-wind" experiments led to the abandonment of the aether concept by some scientists like Emil Cohn or Alfred Bucherer, and to a flurry of efforts to "save" aether by assigning it ever more complex properties by others. Of particular interest was the possibility of "aether entrainment" or "aether drag", which would lower the magnitude of the measurement, perhaps enough to explain MMX results. However, as noted earlier, aether dragging already had problems of its own, notably aberration. A more direct measurement was made in the Hammar experiment, which ran a complete MM experiment with one of the "legs" placed between two massive lead blocks. If the aether was dragged by mass then this experiment would have been able to detect the drag caused by the lead, but again the null result was found. Similar experiments by Hoek placed one leg in a heavy vat of water. The theory was again modified, this time to suggest that the entrainment only worked for very large masses or those masses with large magnetic fields. This too was shown to be incorrect when Oliver Joseph Lodge noted no such effect around other planets.
Another, completely different, attempt to save "absolute" aether was made in the Lorentz-Fitzgerald contraction hypothesis, which posited that everything was affected by travel through the aether. In this theory the reason the Michelson-Morley experiment "failed" was that the apparatus contracted in length in the direction of travel. That is, the light was being affected in the "natural" manner by its travel though the aether as predicted, but so was the apparatus itself, canceling out any difference when measured. Fitzgerald had inferred this hypothesis from a paper by Oliver Heaviside. Without referral to an aether, this physical interpretation of relativistic effects was shared by Kennedy and Thorndike in 1932 as they concluded that the interferometer's arm contracts and also the frequency of its light source "very nearly" varies in the way required by relativity.
Another experiment purporting to show effects of an aether was Fizeau's 1851 experimental confirmation of Fresnel's 1818 prediction that a medium with refractive index n moving with a velocity v would increase the speed of light traveling through the medium in the same direction as v from c/n to:
That is, movement adds only a fraction of the medium's velocity to the light (predicted by Fresnel in order to make Snell's law work in all frames of reference, consistent with stellar aberration). This was initially interpreted to mean that the medium drags the aether along, with a portion of the medium's velocity, but that understanding was rejected after Wilhelm Veltmann demonstrated that the index n in Fresnel's formula depended upon the wavelength of light, so that the aether could not be moving at a wavelength-independent speed. This implied that there must be a separate aether for each of the infinitely many frequencies. This realization tended to undermine belief in the aether as a viable physical concept. Moreover, with the advent of special relativity, Fresnel's equation was shown by Laue in 1907 to be just an approximation, valid for v much smaller than c, for the relativistic formula to add the colinear velocities v (medium) and c/n (rest frame):
Similarly the Sagnac effect, observed by G. Sagnac in 1913 was immediately seen to be fully consistent with special relativity. In fact, the Michelson-Gale-Pearson experiment in 1925 was proposed specifically as a test to confirm the relativity theory, although it was also recognized that such tests, which merely measure absolute rotation, are also consistent with non-relativistic theories.
During the 1920s, the experiments pioneered by Michelson were repeated by Dayton Miller, who publicly proclaimed positive results on several occasions, although not large enough to be consistent with any known aether theory. In any case, other researchers were unable to duplicate Miller's claimed results, and in subsequent years the experimental accuracy of such measurements has been raised by many orders of magnitude, and no trace of any violations of Lorentz invariance has been seen. (A later re-analysis of Miller's results concluded that he had underestimated the variations due to temperature.)
Since the Miller experiment and its unclear results there have been many more experiments to detect the aether. Many of the experimenters have claimed positive results. These results have not gained much attention from mainstream science. For a list and criticisms of those experiments (Kantor, Marinov, Silvertooth, Torr and Kolen, Munera, Cahill) see Physics FAQ: Experiments that Apparently are NOT Consistent with SR/GR.
After some proposals in the 17th and 18th century, Augustin Fresnel in 1818 argued that light moved through a rigid and stationary aether as a way to explain the existence of aberration and polarization.
In 1856 Wilhelm Eduard Weber and Rudolf Kohlrausch performed an experiment to measure the numerical value of the ratio of the electromagnetic unit of charge to the electrostatic unit of charge. The result came out to be equal to the product of the speed of light and the square root of two. The following year, Kirchhoff wrote a paper in which he showed that the speed of a signal along an electric wire was equal to the speed of light. These are the first recorded historical links between the speed of light and electromagnetic phenomena. Meanwhile, James Clerk Maxwell was working on Faraday's lines of force. In his 1861 paper On Physical Lines of Force he modelled these magnetic lines of force using a sea of molecular vortices that he considered to be partly made of aether and partly made of ordinary matter. He derived expressions for the dielectric constant and the magnetic permeability in terms of the transverse elasticity and the density of this elastic medium. He then equated the ratio of the dielectric constant to the magnetic permeability with a suitably adapted version of Weber and Kohlrausch's result of 1856, and he substituted this result into Newton's equation for the speed of sound. On obtaining a value that was close to the speed of light as measured by Fizeau, Maxwell concluded that light consists in undulations of the same medium that is the cause of electric and magnetic phenomena. Maxwell had however expressed some uncertainties surrounding the precise nature of his molecular vortices and so he began to embark on a purely dynamical approach to the problem. He wrote another famous paper in 1864 under the title of A Dynamical Theory of the Electromagnetic Field in which the details of the luminiferous medium were less explicit. Although Maxwell did not explicitly mention the sea of molecular vortices, his derivation of Ampère's circuital law was carried over from the 1861 paper and he used a dynamical approach involving rotational motion within the electromagnetic field which he likened to the action of flywheels. Using this approach to justify the electromotive force equation (the precursor of the Lorentz force equation), he derived a wave equation from a set of eight equations which appeared in the paper and which included the electromative force equation and Ampère's circuital law. Maxwell once again used the experimental results of Wilhelm Eduard Weber and Rudolf Kohlrausch to show that this wave equation represented an electromagnetic wave that propagates at the speed of light, hence supporting the view that light is a form of electromagnetic radiation.
Between 1892 and 1904, Hendrik Lorentz created an electron/aether theory, in which he introduced a strict separation between matter (electrons) and aether. In his model the aether is completely motionless, and it won't be set in motion in the neighborhood of ponderable matter. Contrary to other electron models before, the electromagnetic field of the aether appears as a mediator between the electrons, and changes in this field can propagate not faster than the speed of light. A fundamental concept of Lorentz's theory in 1895 was the "theorem of corresponding states" for terms of order v/c. This theorem states that a moving observer (relative to the aether) in his "fictitious" field makes the same observations as a resting observers in his "real" field. Lorentz noticed that it was necessary to change the space-time variables when changing frames and introduced concepts like physical length contraction (1892) to explain the Michelson-Morley experiment, and the mathematical concept of local time (1895) to explain the aberration of light and the Fizeau experiment. That resulted in the formulation of the so called Lorentz transformation by Joseph Larmor (1897, 1900) and Lorentz (1899, 1904), whereby it was noted by Larmor that the complete formulation of local time is accompanied by some sort of time dilation of moving electrons in the aether.
The work of Lorentz was mathematically perfected by Henri Poincaré who formulated on many occasions the Principle of Relativity and tried to harmonize it with electrodynamics. He declared simultaneity only a convenient convention which depends on the speed of light, whereby the constancy of the speed of light would be a useful postulate for making the laws of nature as simple as possible. In 1900 he interpreted Lorentz's local time as the result of clock synchronization by light signals. And finally in June and July 1905 he declared the relativity principle a general law of nature, including gravitation. He corrected some mistakes of Lorentz and proved the Lorentz covariance of the electromagnetic equations. However, he used the notion of an aether as a perfectly undetectable medium and distinguished between apparent and real time, so most historians of science argue that he failed to invent special relativity.
Aether theory was dealt another blow when the Galilean transformation and Newtonian dynamics were both modified by Albert Einstein's special theory of relativity, giving the mathematics of Lorentzian electrodynamics a new, "non-aether" context. Unlike most major shifts in scientific thought, special relativity was adopted by the scientific community remarkably quickly, consistent with Einstein's later comment that the laws of physics described by the Special Theory were "ripe for discovery" in 1905. Max Planck's early advocacy of the special theory, along with the natural and elegant formulation given to it by Minkowski, contributed much to the rapid acceptance of special relativity among working scientists.
Einstein based his theory on Lorentz's earlier work. Instead of suggesting that the mechanical properties of objects changed with their constant-velocity motion through an undetectable aether, Einstein proposed to deduce the characteristics that any successful theory must possess in order to be consistent with the most basic and firmly established principles, independent of the existence of a hypothetical aether. He found that the Lorentz transformation must transcend its connection with Maxwell's equations, and must represent the fundamental relations between the space and time coordinates of inertial frames of reference. In this way he demonstrated that the laws of physics remained invariant as they had with the Galilean transformation, but that light was now invariant as well. Max Born observed:
Einstein in later years proposed calling empty space equipped with gravitational and electromagnetic fields the "aether", whereby, however, this word is not to denote a substance with its traditional attributes. Thus, in the "aether" there are to be no determinable points, and it is meaningless to speak of motion relative to the "aether." Such a use of the word "aether" is of course admissible, and when once it has been sanctioned by usage in this way, probably quite convenient. From now on aether as a substance vanishes from theory.
With the development of the special relativity, the need to account for a single universal frame of reference had disappeared — and acceptance of the 19th century theory of a luminiferous aether disappeared with it. For Einstein, the Lorentz transformation implied a conceptual change: that the concept of position in space or time was not absolute, but could differ depending on the observer's location and velocity.
Moreover, in another paper published the same month in 1905, Einstein made several observations on a then-thorny problem, the photoelectric effect. In this work he demonstrated that light can be considered as particles that have a "wave-like nature". Particles obviously do not need a medium to travel, and thus, neither did light. This was the first step that would lead to the full development of quantum mechanics, in which the wave-like nature and the particle-like nature of light are both considered to be simplifications of what is "really happening". A summary of Einstein's thinking about the aether hypothesis, relativity and light quanta may be found in his 1909 (originally German) lecture " The Development of Our Views on the Composition and Essence of Radiation"
Lorentz on his side continued to use the aether concept. In his lectures of around 1911 he pointed out that what "the theory of relativity has to say ... can be carried out independently of what one thinks of the aether and the time". He commented that "whether there is an aether or not, electromagnetic fields certainly exist, and so also does the energy of the electrical oscillations" so that, "if we do not like the name of "aether", we must use another word as a peg to hang all these things upon." He concluded that "one cannot deny the bearer of these concepts a certain substantiality".
In the early 1920s, in a lecture which he was invited to give at Lorentz's university in Leiden, Einstein sought to reconcile the theory of relativity with his mentor's cherished concept of the aether. In this lecture Einstein stressed that, in general relativity, space is "endowed with physical quantities"
He pointed out that the aether had been relativized, and thereby lost the last mechanical property that Lorentz had left it, namely, its state of motion. Thus he held that general relativity attributed physical properties to space, including some kind of medium for light, although not a material one. Shortly before his lecture in Leiden in 1920 he admitted in the paper: "Grundgedanken und Methoden der Relativitätstheorie in ihrer Entwicklung dargestellt":
Therefore I thought in 1905 that in physics one should not speak of the aether at all. This judgement was too radical though as we shall see with the next considerations about the general theory of relativity. It moreover remains, as before, allowed to assume a space-filling medium if one can refer to electromagnetic fields (and thus also for sure matter) as the condition thereof.
In later years there have been a few individuals who advocated a neo-Lorentzian approach to physics, but it is Lorentzian only in the sense of positing an absolute true state of rest, which is undetectable and which plays no role in the predictions of the theory. (No violations of Lorentz covariance have ever been detected, despite strenuous efforts.) Hence these theories resemble the 19th century aether theories in name only. For example, the founder of quantum field theory, Paul Dirac, stated in 1951 in an article in Nature, titled "Is there an Aether?" that "we are rather forced to have an aether".
A very small number of physicists (like Dayton Miller and Edward Morley) continued research on the aether into the first decades of the 20th century. However, no evidence of the sort sought by these individuals has ever passed the tests of modern scientific standards. Today the majority of physicists hold that there is no need to imagine that an aether (as a medium for light propagation) exists. They believe that neither Einstein's general theory of relativity nor quantum mechanics have need for positing its existence, that there is no evidence for its existence, and that the assumption of its existence is an unnecessary theory violating the principle of Occam's razor. It is difficult to develop an aether theory consistent with the results of all experiments of modern physics. Some new "aether" concepts have been proposed in recent years, but the descriptions of these concepts differ in fundamental ways from the description of the classical luminiferous aether.
[W]e may say that according to the general theory of relativity space is endowed with physical qualities; in this sense, therefore, there exists an aether. According to the general theory of relativity space without aether is unthinkable; for in such space there not only would be no propagation of light, but also no possibility of existence for standards of space and time (measuring-rods and clocks), nor therefore any space-time intervals in the physical sense. But this aether may not be thought of as endowed with the quality characteristic of ponderable media, as consisting of parts which may be tracked through time. The idea of motion may not be applied to it."