Traditionally, prism-based cameras are often selected over trilinear cameras when it comes to bulk material scanning, as the inherent line-shift of a trilinear camera can only be corrected when the object scan velocity is known. In this article it is demonstrated that the traditional approach is not always necessary if the right trilinear sensor is selected and operated in binning mode to average multiple pixels.
There exist various manufacturing applications where bulk materials are required to be inspected in an automated fashion at high speed. Industrial cameras, especially with line-scan camera technology, are very well suited for the typical high velocity at which the bulk material moves. In the simplest case, a line-scan camera consists of a single linear line of sensor cells. A two-dimensional image similar to an image from an area sensor is created by moving the scan scene perpendicular to the sensor while acquiring successive lines of the image. The single-line sensor can produce a single-channel image, typically in grayscale. However, many bulk material inspection applications require color images. There exist two types of technologies to acquire full-color images:
A single trilinear line-sensor.
A prism-based camera with three single line-sensors.
In the case of the trilinear sensor, the three image lines of a single object point are acquired at different moments in time with the resulting channel-shift in transport direction typically corrected internally. For a perfect correction the scan object velocity needs to be constant and precisely predetermined. In the case of a prism-based camera, the three image channels are acquired at the same moment in time and therefore no extra correction is needed. (Figure 1)
Whenever bulk material is transported on a conveyer belt the object velocity is typically well known. However, there also exist applications for which this is not the case, such as:
Free falling bulk material
The velocity distribution among individual material pieces is rather homogenous and only slightly influenced by air friction.
Bulk material on an inclined plane
The velocity distribution among individual material pieces is influenced by the coefficient of friction of the material and the plane.
For the case of a trilinear line-sensor, unknown object velocity can result in the so-called color fringe or halo effect, for which object contours appear with false color in the transport direction. An example is shown in Figure 2.
However, with the correct selection of a trilinear camera sensor type and proper configuration, it can be equivalent in performance to a prism-based camera in terms of color image quality. We demonstrated this by testing several bulk materials. We did a one-to-one comparison of the trilinear sensor-based camera, — Chromasens allPIXA wave — with an industry-standard prism camera.
As mentioned earlier, trilinear sensor-based cameras can be used as alternatives to prism-based cameras in many conditions. The color fringe effect with trilinear sensors stems from the fact that the color image channels have to be shifted relative to each other by an amount that depends on the physical distance (∆) between the sensor lines (typically R, G, and B) and the magnification β. In sensor datasheets, ∆ is referred to as the pixel-pitch between sensor lines. The physical size of the color shift between two lines (in meters) is βx∆. Typically, the scan speed (vscan) of the object is adjusted to a nominal velocity v0 in such a way that transport resolution and optical resolution are the same (square pixels). The size of the color fringe in pixels is dependent on the ratio of these two velocities and can be calculated using the physical size of a pixel (S) instead of the magnification:
The Chromasens allPIXA wave that was used in testing has a pixel pitch of 10.2 μm and a pixel size of 5.6 μm, resulting in reduced color fringe compared to trilinear cameras that have larger pitches and pixel sizes.
The size of the fringes can be further reduced by increasing the image resolution (smaller β). For the most part standard commercial prism-based line-scan cameras for industrial applications have at most 4096 pixels. Cameras of the Chromasens allPIXA wave family are available with up to 15360 pixels. Accordingly, given the same field of view but more pixels the magnification can be decreased as shown in the following example:
Field of view: 100mm
with 2k (2048) pixel sensor, 48.8μm/ pixel
.with 4k (4096) pixel sensor, 24.4μm/ pixel
with 15k (15360) pixel sensor, 6.5μm/ pixel
Line-shift relative to the camera resolution:
with 4k (4096) pixel sensor, a factor of 2 smaller than for 2048-pixel sensor
with 15k (15360) pixel sensor, a factor of 7.5 smaller than for 2048-pixel sensor
Accordingly, with a 15k trilinear line-scan sensor instead of a 2k trilinear sensor, there will be a factor of 151 less of a color fringe effect with the same field of view.
The remaining color fringe has maximum visibility at an edge where the color changes from black to white in the transport direction. For the trilinear camera tested, the image of the edges in two neighboring color channels is shifted by two pixels relative to each other. An additional trick helps to reduce the fringe further. If multiple pixels are averaged into one single pixel, a process called binning, the shifted edges of all color channels will fall into the same pixel. The remaining shift is now at a sub-pixel level and results at most in a slight discoloration of the edge.
When setting up a trilinear line-scan camera the internal line-shift correction is set to the nominal velocity of the application. The binning approach then only has to correct for velocity variations. Typically, binning windows of 2 to 4 pixels have been useful in this case.
Laboratory Setup and Test Material
Two configurations were tested. The first is the inclined plane setup (see Figure 3, left side). The inclined plane setup is a worse case than the free-falling bulk material, as the velocity distribution of the bulk material particles is larger. The second configuration is the linear stage setup (see Figure 3, right side), in which the bulk material is distributed on a plane that is then moved under the camera. This setup allows adjusting the movement velocity in a controlled fashion. In that way, it is possible to do a one-to-one comparison of individual material particles with both camera types for different velocities.
Bulk Material Used for Testing:
White rice — white objects can be considered as a worst-case scenario, as the color-fringe effect becomes most visible.
Analyzing Velocity Distribution of Bulk Material on Inclined Plane
In conventional line-scan applications with trilinear sensor cameras, the line shift is corrected in the camera for a given global velocity. With free-moving bulk material — inclined plane or free-falling — the adjustment is done for the average velocity, which has to be determined once, when setting up the system.
We analyzed the individual particle speed of bulk material on the inclined plane setup for the case of white rice samples. This was done by blob analysis of the line-shift of each rice kernel of a scanned image. The resulting velocity distribution is illustrated in Figure 5.
We identified a velocity standard deviation of 10%. Assuming our distribution is adequately approximated by a Gaussian distribution we can conclude that 68% of all particles are within the velocity range of +/-10%, and 95% in the range of +/-20% from the average velocity.
One-to-One Comparison of Cameras
Assuming the parameters of the velocity distribution of white rice, we can acquire image data with the linear stage setup at different speeds. For testing, we considered average velocity +/-20%. We performed the experiment for the trilinear and the prism-based cameras using an identical scan scene.
Figure 6 shows a direct comparison of image data.
Comparing the trilinear camera images with native resolution and nominal velocity (right column, 2nd row) with those acquired with +/-20% speed (right column, 1st and 3rd rows), we can observe color fringes at the upper and lower ends of the rice kernel. The zoomed-in image of the upper end of the rice kernel shown in Figure 7 illustrates the effect in more detail.
Binning the native resolution images with 2 × 2 or 4 × 4 pixels removes the effect down to minimal discoloration residue. For the case of 4 × 4-pixel binning, we show zoomed-in versions in Figure 8.
White objects on dark background are the worst-case scenario in terms of the color fringe effect for trilinear cameras. In Figure 9, we illustrate sample images of the other scan objects considered.
The image quality of trilinear and prism-based cameras seems rather similar. The color difference between the cameras stems from differences in the spectral responsivities of the two sensors.
Sample Images of Bulk Material on Inclined Plane
In previous experiments, bulk material was scanned with the linear stage setup for 1-to-1 comparison. A more realistic scenario is acquiring images of bulk material on an inclined plane as shown in Figures 10 and 11.
A comparison of trilinear and prism-based cameras for bulk-material inspection was performed by visual assessment of the color-fringe effect. This effect appears, for instance, when individual particles have distinct velocities. It was demonstrated that selecting a camera with small physical pixel-line distance in combination with high-resolution sensors and pixel-binning can effectively remove visible color-fringes.
Objects for which the reflected light is spectrally flat such as those that appear white in images, exhibit the largest color fringe effect. For other objects, the color fringe effect is generally smaller and for many applications, invisible. If in doubt whether or not color fringe effects are visible in images scanned by a trilinear sensor, it is advisable to test empirically.
There are several advantages to the proposed approach. First of all, there are less expensive trilinear sensors with much higher resolution as compared to prism-based cameras. Further, using a trilinear sensor offers much more flexibility in the selection of stock lenses, as there is no special compensated lens required for the extended optical path of a prism-based camera.
This article was written by Timo Eckhard, Team Leader Research & Innovation, Innovation & IP Management; and Sebastian Georgi, Research & Innovation Manager, Chromasens (Konstanz, Germany). For more information, visit here .