Artificial Vision as it Relates to Robotics and Object Tracking

Today’s blog post is sponsored by DataRay – Advancing the technology of laser beam analysis since 1988

The world of machine learning and artificial intelligence is an evolving field. How close humanity is to a robot takeover is yet to be determined—those conjectures can be left to the dystopian sci-fi movie industry—but in the meantime, let’s explore the current state of artificial vision.

Three of areas that immediately come to mind when considering machine learning and optical recognition are manufacturing, autonomous vehicles, and the defense industry. While these sectors ultimately have different end goals, object recognition is important for autonomous machines. This sort of artificial vision must make it possible to detect, identify, and ‘understand’ what it is being seen. It must be able to identify objects, distinguish them from the background, and properly categorize them. This can be a challenge in many cases, especially when there is reduced contrast between an object and the background. Industrial robots must be able to operate with high precision and accuracy, and in some instances work in conjunction with their human counterparts.

A review out of Spain published in Sensors in 2016 identified a number of common techniques used for machine vision and robot guidance: photogrammetry, structured light, time of fight, and laser triangulation. These methods will be explored further below.

Photogrammetry for Artificial Vision

This method of artificial vision is also known as Image-Based Modeling. This technique uses a camera to take a photo of an object or scene. The field detected by the camera is then split dependent on the camera location. From the photographs, drawings can be created, three-dimensional models produced, or measurements done.  This is accomplished by locating points in the image and finding the intersections of projections of the points. Often, high contrast physical markers are needed on the surface of the object for proper detection to be achieved. In cases where physical markers are not possible or not desirable, the process can be automated using tracking algorithms. Stereo vision is very similar to photogrammetry, though has less of an emphasis on accuracy and extracts information from two viewing locations.

Photogrammetry is used in robotics to help keep track of the robot’s motion. Photogrammetry can be combined with interferometry to create a sort of laser tracking system. The systems work together. For instance, the interferometer may determine the distance a robotic arm may have moved while the photogrammetric portion may measure its angular position. Multiple photogrammetry setups may be used for one robot to determine its position without using interferometry, though this tends to be more expensive and used more for calibration than continuous tracking.

Structured Light Imaging

This method can produce high resolution results. Structured light imaging projects a pattern onto a surface. This pattern is usually modulating. One example of these modulating patterns is a grayscale stripe pattern where the intensity of each pixel is based off a sine wave. This kind of pattern is considered a time-multiplexing technique. After the pattern is projected onto the surface in question, a camera then captures projection. The distortion of the pattern can be related to the shape of the object. In the case of the sine wave pattern, since the phase of the pattern is known, a topography of the surface can be backed out from the resulting image, as the object’s surface will have likely distorted the pattern or introduced a phase shift. With a time-multiplexing pattern, the camera must remain stationary throughout the process. Another version of structured light imaging uses unique patterns and is referred to as a one-shot technique. Using this method, the camera is able to move as each part of the pattern is encoded and unique to a “local neighborhood around it”. An advantage to structured light imaging is that high spatial resolution can be achieved.

Graphic describing the structured light technique for processing objects. Courtesy of Robotics Tomorrow.

Time-of-Flight

This method of artificial vision uses a modulated light source, typically in the visible or infrared wavelength range. The modulated light source may be in a pattern similar to structured light imaging. Both continuous wave and pulsed light can be used for time-of-flight technology. The light is sent out and reflected back to the source. The difference in phase between the light sent out and the light received makes it possible to reconstruct the scene by determining the time it took for the light to make its trip. Ambient light can sometimes decrease the quality of the detected light, increasing the noise level. Time-of-flight sensors can be used in many ways. One way they are used is to help identify defects in products on assembly lines.

Diagram describing the theory behind time-of-flight. Courtesy of Texas Instruments.

An advantage of time-of-flight that has been identified is that that the sensors are “quite compact and lightweight, and allow real-time 3D acquisition with high frame rate”. This makes time of flight very useful for mobile robots whose movement ranges a span greater than that in an instance where photogrammetry could be used. However, the data from the time-of-flight sensors can be limited by the amount of noise, thus requiring careful calibration and/or postprocessing.

Laser Triangulation for Artificial Vision

This method uses a laser and camera placed at known locations to determine the location of a point on an object. Since the location of the laser and camera are known, as well as the distance between them, the object’s location can be calculated. This method requires scanning the surface of the object, as it just locates one point at a time.

Triangulation comes up frequently in production and quality control. It is used in a number of applications from determining if cans are properly vacuum sealed to detecting defects in products on a conveyor belt.