New Microslice Technology for Hyperspectral Imaging

Abstract

We present the results of a project to develop a proof of concept for a novel hyperspectral imager based on the use of advanced micro-optics technology. The technology gives considerably more spatial elements than a classic pushbroom which translates into far more light being integrated per unit of time. This permits us to observe at higher spatial and/or spectral resolution, darker targets and under lower illumination, as in the early morning. Observations of faint glow at night should also be possible but need further studies. A full instrument for laboratory demonstration and field tests has now been built and tested. It has about 10,000 spatial elements and spectra 150 pixel long. It is made of a set of cylindrical fore-optics followed by a new innovative optical system called a microslice Integral Field Unit (IFU) which is itself followed by a standard spectrograph. The fore-optics plus microslice IFU split the field into a large number of small slit-like images that are dispersed in the spectrograph. Our goal is to build instruments with at least hundreds of thousands of spatial elements.

1. Introduction

Hyperspectral imaging is now an established tool within the field of remote sensing, and has seen applications in a wide range of scientific areas, from air and water pollution [1–3] to mineral identification [4,5] and vegetation mapping [6,7]. The ability to obtain high quality spectral data over a contiguous field is also an important requirement in the field of astronomy, where the technique is usually known as integral field spectroscopy [8]. Different types of integral field systems are used in astronomy. The two that are most commonly used are fibre-lenslet integral field units [9,10] in which optical fibres are glued to microlens arrays, and image slicers [11,12] where a series of long thin mirrors “slice” the telescope focal plane into thin images that are reimaged side by side at the entrance slit of a spectrograph. The microslice technology that we are presenting here uses a completely different design.

Every hyperspectral imaging instrument uses some form of multiplexing technique to build the final data cube, usually with two spatial and one spectral dimension. These instruments can be time-multiplexed, as in the pushbroom or Fourier Transform approaches already deployed for several remote sensing satellites, or spatially multiplexed as is the case presented here. Spatial multiplexing methods have the advantage that they collect information in all spatial pixels simultaneously, and can therefore be used to monitor rapidly time-varying phenomena or operate in poor conditions where the spectral transmission of the atmosphere is varying with time.

As an example, we can compare a microslice system with a pushbroom. Instead of the unique slit of the pushbroom, the microslice system would have for example 100 contiguous lines of spatial elements of the same width and length than the slit of the pushbroom. Instead of the scanning pushbroom method, we use a step-and-stare approach in which the instrument points at a field for 100 times longer than would a pushbroom then capturing 100 times more light, then move to the next contiguous field. This translates into the ability to observe at higher spatial and spectral resolution and to observe in fainter illumination conditions. It also permits to observe faint glow at night although there would be a minimum threshold under which the glow is too faint even for our type of instrument. The longer time of observing can also remove the need to use detectors with ultra-high reading times (so use detectors with more pixels for the same price) or reduce the effect of the detector noise with the same type of detector. Compared to a Fourier Transform Spectrometer (FTS), a microslice spectrometer would capture for example one image with spectra 100 pixels long while the FTS would have to make a Fourier scan of 100 images in the same time to get the same bandwidth and spectral resolution. Our instrument also has a large multiplex advantage over Imaging Fourier-Transform Spectrometers (IFTS). For example in [1] an IFTS is described. It has only 128 × 64 = 8,192 spatial elements compared to the 10,000 of our very small prototype. A full scale instrument may very well have 1,000,000 spatial elements which would cover more than 100 times the surface area in the same observing time.

An example of use of the instrument is remote sensing studies of leaf chlorophyll content. Information on the amount and spatial distribution of chlorophyll content is of importance for the study of vegetation productivity and health. We believe that this technology will increase the spatial resolution of the acquired data and provide enough detail for accurate mapping of the leaf chlorophyll content. Another is the detection of pollutants in stream with the goal of removing the need for samplings and further laboratory analysis. With far more light, the instrument could detect much fainter concentrations of the pollutants than other airborne technologies. Also, a higher spectral resolution can be achieved. Generally, the instrument can be used under lower illumination conditions as dusk and dawn, and overcast sky.

Various approaches to spatial multiplexing have been discussed in the context of astronomical instrumentation. Each method has its own advantages and disadvantages, depending on the final application, and the power of each system is ultimately limited by the total number of detector pixels available. The main advantage of the method proposed here is that it delivers a large number of simultaneous contiguous spatial samples, together with reasonable coverage of the spectral domain. The latter can be in the form of wide waveband coverage at low spectral resolution, or higher spectral resolution over a shorter bandpass. All of this can be delivered in a compact, high throughput optical system.

The principle of operation of the microslice hyperspectral imager is shown in Figure 1. First, an image is formed at the input of the microslice system by cylindrical fore-optics. The basic concept is to use in the microslice system a pair of orthogonal cylindrical microlens arrays to segment the input field into a series of rectangular apertures. Each of these apertures is then reformatted by more cylindrical microlens arrays into a thin “microslice” using anamorphic demagnification (so a different demagnification factor in each direction) to get the needed image size in each direction independently. The microslices form virtual slits which feed a conventional transmissive spectrograph, with a collimator/camera subsystem and a dispersion element. The dispersed spectra are tilted to make the most effective use of the available detector pixels and avoid spectral overlap. Each microslice spectrum contains spatial information along (but not across) the slice. The spatial sampling is determined in the horizontal direction of Figure 1 by the width of the segmented apertures while the pixel size determines the spatial sampling along the slices as along the slit of a standard spectrograph. In the case of the prototype system described here, this amounts to ∼10,000 spatial samples that we call “spaxels” for spatial pixels, although our goal is to scale to much larger sizes.

It should be noted that the prototype was developed in order to investigate the performance and characteristics of the microslice hyperspectral imager concept and to explore issues with the assembly and integration. Although this version was not expected to have the full performance available from an optimized design, the goal was to demonstrate a proof of principle, and give some guidance on where improvements in the opto-mechanical design would have the greatest payoff. The mechanical construction was therefore designed to expedite laboratory tests of performance as benchmarked against conventional spectrometers (e.g., an ASD FieldSpec Pro), but also to be robust enough to enable some measurements to be taken in the field. No particular priority was placed on volume or mass.

2. Instrument Design

2.1. Optical Design

The microslice hyperspectral imager is composed of five parts: (i) the anamorphic fore-optics, (ii) the microslice system, (iii) the collimator, (iv) the dispersing element, and (v) the camera (which includes the detector array).

2.1.1. Anamorphic Fore-Optics

The fore-optics in the prototype are designed to view an object 2 m in front of the instrument to facilitate laboratory and field tests, but can be refocused to view an object at infinity. They are made of three cylindrical lenses and a cylindrical doublet comprising two lens assemblies, one imaging the vertical direction the other the horizontal. The lens assemblies have different powers in the directions parallel to, and perpendicular to, the microslice length and two different glasses are used to minimize the chromatic aberrations. The different magnifications in the two orthogonal directions are necessary to create enough space between the microslice images to enable the dispersed spectra to be interleaved on the detector (Figure 1). In our prototype design, the magnification is ∼5× larger in the direction perpendicular to the slitlet length. The input object field is 10 cm × 50 cm giving a total field-of-view of approximately 3° × 15° (as viewed from first lens). The resulting image on the microslice system is approximately 10 mm × 10 mm. The vertical and horizontal input planes of the microslice system are not at the same position along the optical axis so the fore-optics were designed accordingly.

2.1.2. Microslice System (Image Slicer)

The microslice system is the core of the new technology. It is the main part that we wanted to demonstrate with this project. It is made of five cylindrical microlens arrays with pitches of 500 microns or 300 microns. As for the fore-optics, the power of the cylindrical lenslets is either parallel or perpendicular to the slitlet length. To keep the prototype costs low, off-the-shelf all-silica microlens arrays were used (from SUSS MicroOptics) which restricted the choices of focal lengths available. Two identical arrays were used to create an effective lenslet array with a focal length ×2 shorter; if custom microlens arrays were used, the system could be simplified to use just four arrays. The limitation on the choice of focal lengths also meant that the vertical and horizontal image planes could not be superimposed at both input and output. A choice was made to superimpose the output planes (just inside of the substrate of the final lenslet array) and to have the input focal planes separated by 1.5 mm. The microslice system dissects the image at its input into 19 × 31 slices and demagnifies each of them individually. The result is an array of slitlet images each with dimensions ∼130 × 30 μm. A critical characteristic of this system is that the spatial information is retained along each slitlet. One slitlet then gives 17 individual spectra which are adjacent on the detector, one per pixel along the slit, for a total of 310 × 31 spaxels (and hence independent spectra). This is a very significant improvement on previous lenslet array systems and enables far more spectra to be packed onto the detector.

A full explanation of the microslice system concept and design can be found in [13] and more information on the subject in [14,15].

2.1.3. Spectrograph Collimator

The collimator works as in a standard imaging spectrograph. It re-images the slitlets close to infinity and positions the output pupil on the stop of the camera to minimize losses of light by vignetting. This also has the effect of minimizing the effects of parasitic orders from the grism which are cut off at least in part by the camera stop. The grism is not quite in a collimated beam, so that the lens on the camera can be used for fine focus adjustment and the input field is approximately 2.6 m in front of the camera (the induced aberrations are negligible). A custom optical design (5-elements) has been made (Figure 2), since this is the optical system that could the most minimize aberrations.

Inexpensive glasses were used where possible to keep the cost down. All the custom-made and off-the-shelf optics (including the microlens arrays) are anti-reflection coated to reduce light losses, stray light and ghost reflections.

2.1.4. Dispersing Element

The dispersive element is a grism, and is a combination of a 12° prism with a transmission grating operating over a bandwidth of 0.4–0.7 μm. This allows moderate dispersion while maintaining the direction of the optical axis for a compact mechanical design. The grism is placed as near as possible to the camera to be close to its stop. It has 200 lines/mm giving 43 spectral elements of ∼7.0 ±0.4 nm each sampled by ∼3.8 ± 0.2 pixels on the detector. The prism was custom-made and glued to the off-the-shelf transmission grating. The dispersion direction is not exactly in the direction perpendicular to the slitlets, as in a classic spectrograph, in order to avoid spectrum overlaps, but is rotated by 29° (Figure 1).

2.1.5. Camera and Detector

The camera is a commercial Sigma DP2 with 2,640 × 1,760 usable pixels. The lens has a maximum aperture of F/2.8 and a 24 mm focal length. The effective spectral length is, in principle, about 25 slitlet widths after leaving a small dead space around each spectrum. However, the DP2 employs a novel type of CMOS detector (Foveon®), which is made of three-layer pixels (red, green & blue) to give an equivalent of 14.1 Megapixels. We estimated that this property could be used to increase the length of the spectra by ∼70% (after extraction) by allowing the blue end of one spectrum to overlap the red end of another on the same physical pixel. The effectiveness of this approach was however partly compromised by the overlap in the spectral responses of each layer and the proprietary nature of the Sigma raw data format. While the spectra can be theoretically separated even with the overlap, a careful study will be necessary to see if it is possible to do this with the high precision required for remote sensing applications. An alternative would be to use a larger format (e.g., 30 Megapixel) conventional CCD detector to avoid overlaps. For most of the test data (see Section 3) a bandpass-limiting filter (475–650 nm) was used to fully separate the spectra and position the residual zero order in the gaps between them.

2.2. Mechanical Design

A view of the prototype is shown in Figure 3. The complete system with the cover is contained in a space envelope of approximately 0.008 m³ (420 mm L × 150 mm W × 120 mm H) and the optical components and mounts weight ∼5 kg. Considerable further lightweighting could be achieved if required. With some minimal redesigning of the fore-optics, the whole optical train except the camera could fit in a cylinder 32 mm in diameter and 200 mm long, and weight less than a kilogram (the camera weighs 0.25 kg). We believe that this is unmatched for a system capable of delivering ∼10,000 spectra each with at least 25 spectral channels if not 43. The mechanical design of the spectrograph consists of the same five major subassemblies as for the optical design, these being the fore-optics, the microslice system, the collimator, the dispersing element (grism) and the camera/detector.

Each of the optical elements of the fore-optics assembly was mounted into a lens cell using ultraviolet curing optical adhesive, and these cells were then mounted into a cylindrical lens barrel. Setscrews permit rotational alignment of the optical elements within the fore-optics barrel which is necessary because the vertical and horizontal cylindrical optics need to be perpendicular to each other. The same is true for the microslice system. Each of the five microlens arrays were glued to a support after alignment (Section 2.3). The supports were slid sequentially one by one along four dowels and their array glued before the next support was inserted. Finally, the five supports were held altogether with screws. This design permitted to disassemble the system to later place spacers to adjust magnification and superimposed the output horizontal and vertical focuses. Incidentally, it also permitted to correct any errors in the system.

The collimator was custom made so came as one optical component to which the grism assembly, made of the grism and a support to which it was glued, was attached. Collimator-grism assembly, microslice system and fore-optics were each inserted into a kinematic rotation mount with six degrees of freedom thereby permitting all possible alignments. The camera was placed on a three-point mount so could also be adjusted in height and tilt.

All mechanical pieces except for the collimator were designed and machined in house.

2.3. Assembly, Integration and Test

The most critical task during AIT was assembling the microslice system. The five arrays had to be aligned with a high precision including their positions along the optical axis. Assembly was done using a custom jig that holds the microlens array and permits it to move and rotate in the plane of its support (Figure 4). Laser and white light illumination from underneath the nested array was used to measure the position and rotation by comparing the spot patterns produced on a target at 2 m distance with simulations made from ray tracing of the optical design. Using this method, displacements of ∼1 micron could be detected. Once correctly located the arrays were fixed using ultraviolet curing adhesive. An additional problem encountered was that the first array of the stack was not as specified by the manufacturer but had a different focal length; this had to be compensated by reversing and displacing one of the arrays by 1.2 mm along the optical axis using shims. Black masks were used to avoid parasite reflections and to block a few defective microslices that had been damaged whilst the arrays were being anti-reflection coated by the manufacturer. The fore-optics did not require a very high alignment precision and mostly relied on mechanical precision. All of the optical assemblies were then aligned to each other using targets and an alignment telescope, with the microslicer as the reference.

3. Laboratory Performances

Many tests were done on the fully assembled system and also separately on some of its constituent parts. A CCD camera was used to measure the image quality given by the microslice system and by the collimator. Images of spectra were then taken using the fully assembled system illuminated by different continuum and arc lamps. A laser spot was displaced on a white surface on the input focal plane 2-m before the instrument to determine the shape and size of the field of view. Several test patterns were observed to check the spatial imaging performance of the system, although this is currently limited by the reconstruction software under development being used to extract the spectra from the camera and rebuild the 3D data cube (Section 4).

3.1. Microslicer Image Quality

The fore-optics were used to test the image quality of the microslice system, which is a crucial parameter for determining the spatial and spectral resolution of the final instrument. The images at the output (Figure 5) were observed with a 10× microscope objective mounted on a CCD camera. Each pixel on the CCD corresponded to a width of 0.73 ± 0.01 microns on the image plane, permitting precise measurement of the aberrations. Figure 6 shows the intensity distribution across the width of a microslice and its derivative. This gives a good estimate of the spatial Point Spread Function (PSF) at each side of the slice. The measurements show that the PSF width (76% In Slit Energy which is the 1-D equivalent of 50% Encircled Energy) is smaller than a pixel on the spectrograph detector (DP2 camera), which was the design goal. By adding shims between the arrays, it was possible to very precisely superimpose the vertical and horizontal foci; the measured difference in position between the two is less than 11 microns. A second test was made by imaging a series of black and white striped patterns (test targets) with widths ranging from 2 pixels to 0.5 pixel. Again this confirmed the as-built image quality of the slicer alone is <1 pixel.

3.2. Full Instrument Tests

The test of the full instrument permits to evaluate the combined effects of all the optical aberrations from the fore-optics to the camera. Note that individual tests of the collimator and the fore-optics were also made confirming their good quality.

3.2.1. White Light Tests with Uniform Illumination

The first tests of the full system were made with a white light continuum source. These showed that the spectra were well formed (Figure 7) with sharp edges, and enabled a first (approximate) measurement of the detector response for each color as a function of wavelength. A filter of bandwidth 475 nm to 650 nm was used to restrict the overlap between spectra and simplify the problem of dealing with contamination from the zero order by placing its images in the gap between spectra. The array of dispersed first order spectra from the 19 × 31 array of horizontal microslices is clearly seen. The offset array of single yellow lines are the images of the zero order (one per microslice) which must be removed in software. The zoomed section shows that the spectra do not overlap with a 475–650 nm filter, and that the contamination from the zero order is minimal in this range. The black section in the lower left was masked because of a defect on one of the lenslet arrays caused during anti-reflection coating.

Figure 8 shows the wavelength response of the three layers (labeled red, green, blue) of the Foveon sensor in the Sigma DP2 obtained from the spectra (note that the wavelength calibration is approximate using the cut-off wavelengths on each side). The three colors are sufficiently well separated to be able to have superimposed spectra over the full 400 nm to 700 nm range of the system and still separate them using the different color responses of the layers. However a careful study will be necessary to see if it is possible to do this with the high precision required for remote sensing applications.

3.2.2. Spatial Line Spectra

To test the overall spatial resolution of the instrument (including the spectrograph), a target made of black and white parallel lines with a middle white stripe was constructed; the images of the lines should be 1.7 pixels wide on the detector. The pattern can clearly be seen in the dispersed spectral image (Figure 9(left)) which shows that the system has good spatial resolution at all wavelengths. To make a quantitative estimate, the contrast between lines at many positions in the field and at three colors was measured. Figure 9(right) shows a profile measured in a single column across 6 different spectra. Because of the stagger in the spectra, the corresponding color (wavelength) for the lefthand profile is red; the next is green and the next blue, then again red-green-blue. Nine of these cuts distributed equally over the whole field were used. The average contrast ratio between minimum and maximum intensity is 0.31. Assuming that the PSF is a gaussian, the FWHM of a point source image giving this contrast value can be calculated to be 1.5 ± 0.2 pixels, which is better than our goal of two pixels FWHM at the detector (end to end).

3.2.3. Spectral Line Spectra

The spectral calibration of the instrument was examined by illuminating the system with a series of different spectral lamps (Figure 10). Because these elements have precise emission lines in the 475–650 nm wavelength range, the images can be used to spectrally calibrate the instrument and also to determine its spectral resolution. Example spectra extracted from the images are shown in Figure 10. The spectral resolution (FWHM) obtained from these spectra is approximately 7 nm which corresponds to the theoretical resolution of the system. This means that, while the edges of the spectral elements are blurred, the FWHM remains the same. Note that the spectrum extraction software is a work in progress that still needs fine tuning.

4. Data Processing

Further details of data processing and additional results will be presented in future papers as [16]. The raw spectra from the DP2 camera are packed closely into a 2D frame as shown in Figure 7. Each of these spectra is from a different 1D slice in the original image plane, and retains spatial information along that slice. Information in this form is of course difficult to interpret easily, so a set of simple MATLAB scripts were developed to create a 3D spectral data cube from the data, with two spatial dimensions preserving the original spatial coverage of the input image and one spectral dimension. Once the data cube is created, a spectrum can be extracted using standard software such as ENVI for any specified spatial pixel in the cube and subsequently processed (e.g., to remove instrumental effects or perform a wavelength calibration).

Figure 11 shows the operations whereby a spectral image obtained from the Sigma DP2 camera is rearranged to form the datacube. The microslicer produces 31 × 19 slices (31 × 18.5 in fact since some on one edge are truncated), each of which produces an independent dispersed 2D spectrum with 17 spatial pixels and covering a wavelength range of 475–650 nm. This means that the data can be converted into a full datacube of spatial dimensions 310 × 31, with an arbitrary number of spectral bands.

In creating the datacube, the first step is to use a geometric transformation to rectify the dispersed spectra, which are dispersed at an angle of 29 degrees relative to the CCD rows due to the rotation angle of the grism. After this rectification procedure, the spectral data cube is rebuilt slice by slice using the known relative positions of the microslices. As can be seen in Figure 7, there is a certain amount of geometric distortion which should be accounted for in a full reduction procedure. There are also “zero order” spectra produced by the grism; given the known first and second order response curves for the grism, the zero order contamination can in principle be modeled and subtracted from the data cube. Actually, the zero order spectra are almost completely in the gaps between spectra so they can be easily removed.

The code was given a preliminary test consisting of an image of a leaf on a black background. Figure 12 shows the reconstructed image of a leaf from the spectra. There are some clear inaccuracies but the leaf is still very much recognizable. The total reflectance on the left is obtained by integrating the whole spectrum on each point of the image. On the right, each spectrum was separated into three, corresponding to the limits of red, green and blue, and then each sub-spectrum was integrated to get the RGB colors. A close inspection shows the spatial sampling. The image is made of rectangles that are about two times higher than wide. The horizontal sampling is determined by the pixels on the detector along the slices while the vertical sampling is determined by the slice width in the microslice system. Lines made of a series of rectangles are apparent especially the one following a 45° diagonal on the bottom right. These are veins of the leaf. The bright rectangles are where the vein is centered in the vertical direction. At the intermediate positions between these rectangles, the vein is at the edge of the rectangles in the vertical direction and its light is split between two of them.

A further test was performed to determine the instruments performance in a more practical situation. Measurements were taken of a freshly cut ivy leaf every day for 6 days to record how its reflectance spectrum changed over the drying period. The measurements were taken both with our spectrometer and with a commercial ASD FieldSpec 3® point spectrometer, to provide a comparison. The results of this experiment are shown in Figure 13. As can be seen, the general shapes of the leaf spectra are very similar for both instruments, and the reflectance values are virtually identical. Typical differences between the two are of the order of 0.001–0.002. Our data shows a slight ripple which we hope to remove using more sophisticated data extraction algorithms.

5. Conclusions

We have designed and built a compact hyperspectral imaging camera which utilizes a new form of image slicing based on cylindrical microlens array optics. This prototype utilizes a method that astronomical instruments use to capture information, but in a compact and usable way appropriate for earth observation applications. From the conceptual design to the physical construction of this prototype there were a lot of issues for us to tackle especially because there was many new technological firsts. For example it is the first time that a microslice system is designed, built and used. Previous microlens systems in astronomy used one microlens array with hexagonal microlenses while ours uses five arrays with cylindrical microlenses. This results in five times more spatial elements for the same spectrograph and spectral length. The fore-optics are also highly unusual because they are made of cylindrical lenses and because their output has a different focal plane position along the optical axis for the vertical and horizontal directions.

Our proof-of-concept prototype has been subject to extensive tests in the laboratory and the field, and is meeting most of its key performance requirements. The laboratory results that we presented in this paper demonstrated a promising instrument for remote sensing applications even though there is still ground for improvement. We demonstrated that the image quality at the output of the microlens system and on the detector was excellent especially we obtained an image quality better than our goal of two pixels along the length of the slice images on the detector. This demonstrated the most critical capability of our instrument, that the spatial information is maintained along the slice images. The strength of the instrument is evidently to have a much larger number of spatial elements than pushbroom spectrographs and imaging Fourier-Transform spectrometers. This is done while delivering full spectra on the detector contrary to filter imaging systems and non-imaging Fourier-Transform spectrographs. The system permits to capture far more light which translates into observing under fainter illumination as dawn and dusk, higher spatial and spectral resolutions and probably observation of faint glow at night. The instrument is at its best when used to monitor rapidly time-varying phenomena or operate in poor conditions where the spectral transmission of the atmosphere is varying with time. Furthermore, its compact size allows easy transport and installation on aerial or ground platforms and lots of potential for installation on small UAV platforms.

Areas for improvement include replacing the DP2 camera with a fast-readout CCD and new custom camera optics. The images of the DP2 camera happen to be difficult to analyze because it was impossible to acquire critical information from the manufacturer. Other areas are upgrading the microlens image slicer with new better custom lenslet arrays, and developing more sophisticated data extraction and processing routines. We also plan to continue to deploy the instrument for further field tests and to characterize its long term stability with reference to single point field spectrometers.