Introduction
A spectral imager is a hybrid camera that is one-part camera and one-part spectrometer. As with a digital camera, you obtain a 2D image, but the spectrometer channel adds a third dimension. The “image” that you record with a spectral imager is a 3D data-cube, where the third dimension contains spectral information. In other words, each pixel in the image contains the spectrum of light emitted or reflected from the object or light source.
To understand spectral imaging, first consider how a colour camera works. A standard colour camera typically employs a Bayer pattern of red, green and blue bandpass filters applied directly onto the image sensor to form a colour image. Simplistically, you can regard one third of the sensor’s pixels as filtered green, one third blue and one third red. Software interpolation of the separate RGB pixels is used to recreate a full frame colour image. The spectral information contained in a colour image is limited to the amount of visible light which passes through the bandpass RGB filters. In other words, this is not spectral data per se, rather an image which correlates to some extent with the colour of the object that we would perceive looking directly at it.
The RGB colour camera represents a multispectral imager in its simplest form, using three spectral bands for collecting light in the red, green and blue wavelength regions. However, most multispectral cameras have additional or different filtered bands that enable other information to be interpreted from an image. Many satellite imaging cameras, such as Landsat, use multispectral discrete wavebands to image the Earth to discriminate vegetative and crop types, determine soil condition, map geological features, assess coral reef health and much more.
Hyperspectral imaging takes the concept to the next level where each pixel contains much richer spectral information.
Multispectral Versus Hyperspectral Imaging
From the outset it is useful to define the difference between multispectral and hyperspectral imaging.
The distinction is often based on the number of wavebands detected by a camera with multispectral cameras having maybe 3 to 15 bands and hyperspectral cameras having more than 100 bands. However, a better distinction is that hyperspectral imaging uses continuous and contiguous ranges of wavelengths whilst multispectral imaging uses a subset of wavebands which are usually broader and located at specific regions of the spectrum. Generally, multispectral images do not produce the spectrum of an object but measure at discrete wavebands instead.
Recent advances in sensor design, image processing speed, machine learning and artificial intelligence have cleared the path for a wide range of applications employing spectral imaging, ranging from satellite based/airborne remote sensing and military target detection to industrial quality control and lab applications in medicine and biophysics. Due to the rich information contained in multispectral and hyperspectral images, they are uniquely well suited for automated image processing, whether it is for online industrial monitoring or for remote sensing.
Numerous technical approaches have been used to implement spectral imaging. This article will review some of those outlining the mode of operation and the relative merits of each.
Multispectral Imaging
Several approaches for implementing multispectral imaging have been adopted over the years including simple filter wheel techniques and imaging systems comprising of multiple cameras with individual bandpass filters. At Pro-Lite we generally offer multispectral cameras employing mosaic filter patterns overlaying 2D image sensors. These cameras provide a limited spectral dataset but are robust, with no moving parts, and have a distinct advantage to many hyperspectral cameras in terms of measurement speed. These multispectral cameras can generally be regarded as real-time “snapshot” cameras allowing for the capture of rapidly changing objects at video rates.
Filter-on-Chip (Mosaic) Multispectral Imaging
Filter arrays can be manufactured so they are deposited and patterned directly on top of image sensor pixels. These filters are usually interference-based filters manufactured using semiconductor process technologies. These are typically bandpass filters or Fabry-Perot filters.
Multispectral imagers based on these filter-on-chip technologies sample from spectral bands. They provide fast spectral imaging and can have integrated classification software for real-time identification of materials. These cameras can be made in volume with pre-configured filter bands for specific applications. The manufacturing process is easily scalable and can lead to high-volume and low-cost sensors. If the wavebands are not optimal for a specific application, then custom filter arrangements can be configured but then there will be associated design and manufacturing costs. Spatial resolution is often less than some of the hyperspectral cameras as the resolution of the integrated image sensor is divided by the number of spectral bands in the mosaic pattern.
Imec have produced a range of multispectral sensors based on their semiconductor fab expertise. Cameras operating in the VIS/NIR wavelength range have been available for a few years but more recently they have introduced mosaic cameras working in the SWIR wavelength range.
Related product: Snapshot mosaic SWIR 3×3 and 4×4 cameras from Imec
Plenoptic or Light-Field Multispectral Imaging
Unlike conventional cameras, a plenoptic camera has an additional array of micro lenses located in front of the image sensor. Each micro lens projects a micro image on the sensor plane. A multispectral imager based on the plenoptic or light-field design also includes a filter array at the entrance of the camera.
The LightShift camera from Surface Optics has a 4 x 4 filter array in an interchangeable «tray» placed at the entrance to the camera. The microlens array projects 16 filter images from this filter tray onto the image sensor forming a mosaic of super-pixels on the sensor. As a result the camera produces real-time, video-rate capture at 16 wavebands (spectral channels). One advantage of this approach is that the filter tray can be swapped to allow the selection of application-specific filters or polarisers.
Related product: LightShift™ LVIRA from Surface Optics Corporation
Hyperspectral Imaging
Compared to multispectral cameras, hyperspectral cameras provide much richer spectral datacubes. For each pixel in an image, a hyperspectral camera acquires the light intensity (radiance) for a large number (typically high tens to several hundred) of contiguous spectral bands. Every pixel in the image thus contains a continuous spectrum (in radiance or reflectance) and can be used to characterise the objects in the scene with great precision and detail. Hence, hyperspectral imaging leads to a vastly improved ability to classify the objects in the scene based on their spectral properties.
Many approaches have been explored to produce hyperspectral imaging systems. The simplest approach is perhaps the Whisk-Broom method, where a mirror scans a surface and light is reflected into a single point spectrometer, which collects data one pixel at a time to slowly build up a 3D hypercube.
Push-Broom (Linescan) Hyperspectral Imaging
In a traditional push-broom hyperspectral camera one narrow spatial line in the scene is imaged at a time, and this line is split into its spectral components before reaching the 2D sensor array. A line of spatial information is imaged across the row of pixels and the spectral information for each point on that line is dispersed along each pixel column. Then, as the camera moves across the scene, for example, by means of aircraft or conveyor belt movement, a hypercube is produced with 2D spatial information and the spectral information in the 3rd dimension. In the final image every spatial pixel in the image contains a full spectrum.
HySpex manufacture high quality push-broom hyperspectral cameras. The camera fore-optic images the scene onto a slit. After collimation, a transmission grating separates the different wavelengths and the light is then focused onto the 2D detector array. Utmost care is taken during the manufacturing process to ensure the optics are optimally aligned to provide the highest quality hyperspectral imaging data.
Related product: HySpex cameras from NEO
As explained, with traditional push-broom systems either the camera has to move (on a UAV, aircraft, rotating tripod head or translation stage) or the object itself has to move under a stationary camera (for example, for objects moving on a conveyor belt). More recently, internal scanning Push-Broom cameras have been developed where the scanning mechanism is built into the camera body (the mechanical system moves the spectrometer entrance slit internally). With these cameras there is no need for an external translation stage or moving conveyor belt, instead, the camera can just remain static. This can be useful when space is limited and it also allows the camera to be easily attached to a microscope for hyperspectral microscopy imaging applications.
Tunable Bragg Filter Hyperspectral Imaging
A different approach to creating a hyperspectral image is to use electronically tuneable filters that are mounted in front of a monochrome camera to produce a stack of images at a sequence of wavelengths, forming the familiar image cube. Various filter technology has been used including liquid crystals, acousto-optic filters and Fabry Perot interferometers.
Pro-Lite work with Photon etc who manufacture hyperspectral imagers based on Bragg Tuneable gratings. These diffraction gratings can be used in transmission or reflection mode to extract a narrow band of wavelengths for imaging. Photon use this technology in their hyperspectral microscopy platform to produce hyperspectral hypercubes with both high spatial and high spectral resolution.
Related product: IMA and RIMA from Photon etc
Fourier Transform Hyperspectral Imaging
Fourier-transform (FT) spectroscopy is a novel approach for hyperspectral imaging. The technique uses interference of light rather than dispersion to measure spectra. Light entering the camera is split into two optical paths within an interferometer. The Fourier Transform of the resulting interferogram yields the spectrum of the light entering the camera. The camera has a large entrance aperture which results in a high throughput (high sensitivity), high signal-to-noise camera. FT-based imagers are a good solution for low light level applications.
Related product: Hera from Nireos
Snapshot Hyperspectral Imaging
The hyperspectral techniques mentioned so far require time to spatially scan a scene or sequence through wavelengths to create an image hypercube. However, there are some imaging technologies that enable hyperspectral snapshot imaging where all the spectral data at every pixel in an image is captured instantaneously.
Multi-Point Spectrometer Hyperspectral Imaging
.jpg)
The FireflEYE hyperspectral camera from Cubert employs a novel multi-point spectrometer technology, which strikes a balance between spectral resolution and spatial resolution. The camera is based on a 50 x 50 microlens array together with a prism for light dispersion. The resulting camera produces an image size of 50 x 50 pixels each with 125 spectral channels with all 2500 spectra recorded simultaneously. With only 50×50 pixels it is difficult to resolve spatial structures in some images. Therefore, the FireflEYE contains a second, panchromatic (monochrome) sensor that provides a higher resolution (1000 x 1000 pixel) co-registered image of the same view. Pan-sharpening techniques can be used to combine the spatial and spectral data from the two integrated sensors.
Related product: FireflEYE from Cubert
Plenoptic or Light-Field Hyperspectral Imaging
Cubert have recently adapted the plenoptic camera technology to produce a snapshot hyperspectral camera with increased spatial resolution. This camera uses a combination of a continuously variable bandpass filter, a microlens array and an Ultra-HD CMOS sensor with 20 Megapixel resolution. The result is an image resolution of 400 × 400 pixels, each of which having 100 spectral channels. The spectra are captured in real-time at video rates and provide an excellent system for high resolution imaging dynamically changing scenes.
Related product: Ultris from Cubert
