In Computer Security, a covert channel is a type of computer security attack that creates a capability to transfer information objects between processes that are not supposed to be allowed to communicate by the computer security policy. The term, originated in 1972 by Lampson  is defined as "(channels) not intended for information transfer at all, such as the service program's effect on system load." to distinguish it from Legitimate channels that are subjected to access controls by COMPUSEC.
A covert channel is so called because it is hidden from the access control mechanisms of ultra high assurance secure operating systems since it does not use the legitimate data transfer mechanisms of the computer system such as read and write, and therefore cannot be detected or controlled by the hardware based security mechanisms that underly ultra high assurance secure operating systems. Covert channels are exceedingly hard to install in real systems, they can often be detected by monitoring system performance, they are very noisy (very high signal to noise ratio) and very low data rates (few bits per second). They can also be removed manually with a high degree of assurance from secure systems by well established covert channel analysis strategies.
Covert channels are distinct from, and often confused with legitimate channel exploitations that attack low assurance pseudo-secure systems using schemes such as steganography or even less sophisticated schemes to disguise prohibited objects inside of legitimate information objects. These legitimate channel data hiding schemes are specifically not covert channels and are prevented by ultra high assurance secure OSs.
Covert channel analysis is the only proven way to control covert channels, however secure operating systems can easily control legitimate channels. Distinguishing these is important. Analysis of legitimate channels for hidden objects is often misrepresented as the only successful means to control disallowed flow in legitimate channels. However, this amounts to analysis of large amounts of software, widely known since 1972 to be unsuccessful . Without being informed of this, some are misled to believe an analysis will "manage the risk" of these legitimate channels.
Lampson's definition of a covert channel was paraphrased in the TCSEC  specifically to refer to ways of transferring information from a higher classification compartment to a lower classification. In a shared processing environment, it is difficult to completely insulate one process from the effects another process can have on the operating environment. A covert channel is created by a sender process modulats some condition (such as free space, availability of some service, wait time to execute) that can be detected by a receiving process.
The TCSEC defines two kinds of covert channels:
The TCSEC, also known as the Orange Book, requires analysis of covert storage channels to be classified as a B2 system and analysis of covert timing channels is a requirement for class B3.
The possibility of covert channels cannot be completely eliminated, although it can be significantly reduced by careful design and analysis.
The detection of a covert channel can be made more difficult by using characteristics of the communications medium for the legitimate channel that are never controlled or examined by legitimate users. For example, a file can be opened and closed by a program in a specific, timed pattern that can be detected by another program, and the pattern can be interpreted as a string of bits, forming a covert channel. Since it is unlikely that legitimate users will check for patterns of file opening and closing operations, this type of covert channel can remain undetected for long periods.
A similar case is port knocking. In usual communications the timing of requests is irrelevant and unwatched. Port knocking makes it significant.
As Handel and Sanford take a broader perspective and focus on covert channels within the general design of network communication protocols. They employ the OSI (Open System Interconnection) as a basis for their development in which they characterize system elements having potential to be used for data hiding. The adopted approach has advantages over these are because standards opposed to specific network environments or architectures are considered. Foolproof stenographic schemes are not devised.
Rather, basic principles for data hiding in each of seven OSI layers are established. Besides suggesting the use of the reserved fields of protocols headers (that are easily detectable) at higher network layers, Handel and Sanford also propose the possibility of timing channels involving CSMA/CD manipulation at the physical layer.
The work by them identifies covert channel merit such as:
The covert channel analysis presented here, however does not consider issue such as interoperability of these data hiding techniques with other network nodes, covert channel capacity estimation, effect of data hiding on the network in terms of complexity and compatibility. Moreover, the generality of the techniques cannot be fully justified in practice since the OSI model does not exist per se in functional systems.
As Girling first analyzes covert channels in a network environment. His work focuses on local area networks (LANs) in which three obvious covert cannels (two storage channel and one timing channel) are identified. This demonstrates the real examples of bandwidth possibilities for simple covert channels in LANs. For a specific LAN environment, the author introduced the notion of a wiretap per who monitors the activities of a specific transmitter on LAN. The covertly communication parties are the transmitter and the wire trapper. The covert information according to Girling can be communicated through any of following obvious ways:
I. By observing the addresses as approached by the transmitter.
If total number of addresses, a sender can approach is 16, then
there is a possibility of secret communication having 4 bits for
the secret message. The author termed this possibility as covert
storage channel as it depends in what is sent (i.e.. which address
is approached by the sender)
II. In the same way, the other obvious storage covert channel would depend on the size of the frame sent by the sender. For the 256 possible sizes, the amount of covert information deciphered from one size of the frame would be of 8 bits. Again this scenario was termed as the covert storage channel.
III. The third scenario presented is pertaining to the existence sends can be observed by the wire trappers to decipher for instance “0” for the odd time difference and “1” for the even time difference.
The scenario transmits covert information through “a when-is–sent” strategy therefore termed as timing covert channel. The time to transmit a block of data is calculated as function of software processing time, network speed, network block sizes and protocol overhead. Assuming block of various sizes are transmitted on the LAN, software overhead is computed on average and novel time evaluation is used to estimate the bandwidth (capacity) of covert channels are also presented. The work paves the way for future research.
A more specific approach is adopted by Rowland. Focusing on the IP and TCP headers of TCP/IP Protocol suite, Rowland devises proper encoding and decoding techniques by utilizing the IP identification field, the TCP initial sequence number and acknowledge sequence number fields. These techniques are implemented in a simple utility written for Linux system running version 2.0 kernels.
Rowland simply provides a proof of concept of existence as well as exploitation of covert channels in TCP/IP protocol suite. This work can, thus, be regarded as a practical breakthrough in this specific area. The adopted encoding and decoding techniques are more pragmatic as compared to previously proposed work. These techniques are analyzed considering security mechanisms like firewall network address translation.
However, the non-detectability of these covert communication techniques is questionable. For instance, a case where sequence number field of TCP header is manipulated, the encoding scheme is adopted such that every time the same alphabet is covertly communicated, it is encoded with the same sequence number.
Moreover, the usages of sequence number field as well as the acknowledgment field cannot be made specific to the ASCII coding of English language alphabet as proposed, since both fields take in to account the receipt of data bytes pertaining to specific network packet(s).
The Data Hiding in TCP/IP Protocol suit by Covert channels have following important aspects: