A time-of-flight camera (TOF camera) is a camera system that creates distance data with help of the time-of-flight (TOF) principle which is different from time-of-flight mass spectrometry. The principle is similar to that of LIDAR scanners with the advantage that whole scene is captured at the same time.
Time-of-flight cameras are relatively new devices, as the semiconductor processes have only recently become fast enough for such devices. The systems cover ranges of a few meters up to about 60 m. The distance resolution is about 1 cm. The lateral resolution of time-of-flight cameras is generally low compared to standard video cameras, at 320 × 240 pixels or less.[1][2][3][4][5] Only one camera reports 484 x 648 pixels of resolution using a standard CCD sensor.[6] The biggest advantage of the cameras may be that they provide up to 100 images per second.
Contents |
Several different technologies for time-of-flight cameras have been developed.
There are devices with a pulsed laser and a custom imaging integrated circuit with a fast counter behind every pixel. These devices produce depth values for each pixel on every frame. Typical image sizes are 128 x 128 pixels. Ranges up to 22,000 feet with an eye-safe narrow beam have been achieved. Detectors are typically InGaAs (indium-gallium-arsenide) devices.[7]
Photonic Mixer Devices (PMD) [8] and the Swiss Ranger works by modulating the outgoing beam with an RF carrier, then measuring the phase shift of that carrier on the receive side. This is a compact, short-range device. This approach has a modular error challenge; ranges are mod the maximum range, which is the RF carrier wavelength. With phase unwrapping algorithms, the maximum uniqueness range can be increased. The Swiss Ranger has ranges of 5 or 10 meters, with 176 x 144 pixels. The PMD can provide ranges up to 60m. Illumination is pulsed LEDs, rather than a laser. The demodulation is usually achieved by gating the sensor in synchrony with the light source modulation, so in essence they are range gated imagers.[9]
This is the most promising technology, invented by Antonio Medina. The phase detector is the gate or shutter in the camera. The gate allows collection of portions S2 and S1 of the received light pulse S. The portions are dependent of the time of arrival, and range is derived from them according to Medina's equation, z =R(S2-S1)/2S+R/2 for an ideal camera. R is the camera range, determined by the round trip of the light pulse.[6][10]
The Z-cam,[11] and Canesta 3D cameras are range-gated systems.
Similar principle is used in the ToF camera line developed by Fraunhofer Institute of Microelectronic Circuits and Systems and TriDiCam. These cameras employ photodetectors with fast electronic shutter used as a gated integrator and pulsed lasers.
There are also "range gated imagers", which are not 3D cameras. The gate is open to collect the totality of the reflected pulse and nothing else. Anything outside a specified distance range can be suppressed. Those are useful for seeing through fog. A pulsed laser provides illumination, and an optical gate allows light to reach the imager only during the desired time period.[1] [12]
A time-of-flight camera consists of the following components:
The simplest version of a time-of-flight camera uses light pulses. The illumination is switched on for a very short time, the resulting light pulse illuminates the scene and is reflected by the objects. The camera lens gathers the reflected light and images it onto the sensor plane. Depending on the distance, the incoming light experiences a delay. As light has a speed of c = 300,000,000 meters per second, this delay is very short: an object 2.5 m away will delay the light by:
The pulse width of the illumination determines the maximum range the camera can handle. With a pulse width of e.g. 50 ns, the range is limited to
These short times show that the illumination unit is a critical part of the system. Only with some special LEDs or lasers is it possible to generate such short pulses.
The single pixel consists of a photo sensitive element (e.g. a photo diode). It converts the incoming light into a current. In analog timing imagers, connected to the photo diode are fast switches, which direct the current to one of two (or several) memory elements (e.g. a capacitor) that act as summation elements. In digital timing imagers, a time counter, running at several gigahertz, is connected to each photodetector pixel and stops counting when light is sensed.
In the diagram of an analog timer, the pixel uses two switches (G1 and G2) and two memory elements (S1 and S2). The switches are controlled by a pulse with the same length as the light pulse, where the control signal of switch G2 is delayed by exactly the pulse width. Depending on the delay, only part of the light pulse is sampled through G1 in S1, the other part is stored in S2. Depending on the distance, the ratio between S1 and S2 changes as depicted in the drawing.[14] Because only small amounts of light hit the sensor within 50 ns, not only one but several thousands pulses are sent out (repetition rate tR) and gathered, thus increasing the signal to noise ratio.
After the exposure, the pixel is read out and the following stages measure the signals S1 and S2. As the length of the light pulse is defined, the distance can be calculated with Medina's formula:
In the example, the signals have the following values: S1 = 0.66 und S2 = 0.33. The distance is therefore:
In the presence of background light, the memory elements receive an additional part of the signal. This would disturb the distance measurement. To eliminate the background part of the signal, the whole measurement can be performed a second time with the illumination switched off. If the objects are further away than the distance range, the result is also wrong. Here, a second measurement with the control signals delayed by an additional pulse width helps to suppress such objects. Other systems work with a sinusoidally modulated light source instead of the pulse source.
In contrast to stereo vision or triangulation systems, the whole system is very compact: the illumination is placed just next to the lens, whereas the other systems need a certain minimum base line. In contrast to laser scanning systems, no mechanical moving parts are needed.
It is very easy to extract the distance information out of the output signals of the TOF sensor, therefore this task uses only a small amount of processing power, again in contrast to stereo vision, where complex correlation algorithms have to be implemented. After the distance data has been extracted, object detection, for example, is also easy to carry out because the algorithms are not disturbed by patterns on the object.
Time-of-flight cameras are able to measure the distances within a complete scene with one shot. As the cameras reach up to 100 frames per second, they are ideally suited to be used in real-time applications.
Although most of the background light coming from artificial lighting or the sun is suppressed, the pixel still has to provide a high dynamic range. The background light also generates electrons, which have to be stored. For example, the illumination units in today's TOF cameras can provide an illumination level of about 1 watt. The Sun has an illumination power of about 50 watts per square meter after the optical bandpass filter. Therefore, if the illuminated scene has a size of 1 square meter, the light from the sun is 50 times stronger than the modulated signal.
If several time-of-flight cameras are running at the same time, the cameras may disturb each others' measurements. There exist several possibilities for dealing with this problem:
In contrast to laser scanning systems, where only a single point is illuminated at once, the time-of-flight cameras illuminate a whole scene. Due to multiple reflections, the light may reach the objects along several paths and therefore, the measured distance may be greater than the true distance.
Time-of-flight cameras are also used in assistance and safety functions for advanced automotive applications such as active pedestrian safety, precrash detection and indoor applications like out-of-position (OOP) detection.[15][16] [17]
As time-of-flight cameras provide distance images in real time, it is easy to track movements of humans. This allows new interactions with consumer devices such as televisions. Another topic is to use this type of cameras to interact with games on video game consoles.[18]
Other applications are measurement tasks, e.g. for the fill height in silos. In industrial machine vision, the time-of-flight camera helps to classify objects and help robots find the items, for instance on a conveyor. Door controls can distinguish easily between animals and humans reaching the door.
Another use of these cameras is the field of robotics: Mobile robots can build up a map of their surroundings very quickly, enabling them to avoid obstacles or follow a leading person. As the distance calculation is simple, only little computational power is used.
 FOTONIC-B70 by Fotonic |
 SwissRanger 4000 by MESA Imaging |
 PMD[vision] CamCube by PMDTechnologies |
 USB-powered TOF camera out of the European ARTTS project |
 USB-powered single board PMD camera |
  | Video on YouTube: Robot control by 3D-object tracking - YouTube - Human-Robot-Interface contactless with 3D-Camera |
| | Mesa Imaging - Mesa Imaging AG - SwissRanger SR4000 - miniature 3D time-of-flight range camera. 3D Camera |
|
|
Wikis
Quiz
Map
Search
