Created by Thomas Ostersen
This blogpost was motivated by an invitation to present at a recent Geological Survey of Queensland (GSQ) webinar on advances in remote sensing. Time constraints prevented us from presenting, so in lieu of a talk we’ve put together this blog and an accompanying public github repository with all the code used in data processing and visualisation. The main goal of this blog and code repository is to demonstrate some fully open source methods and workflows for manipulating airborne hyperspectral data sets with relatively modest computing resources.
GSQ HyMap Surveys
Between 2006 and 2007 HyVista Corporation flew a series of HyMap™ airborne hyperspectral surveys as part of the Next Generation Mineral Mapping initiative, a collaboratory project involving the GSQ, CSIRO, Geoscience Australia and James Cook University. The project sought to deliver areally extensive, high resolution mineral mapping products as pre-competitive data sets for the Australian resources community in prospective areas of northern Queensland (Cudahy et al., 2008). All mineral mapping products and other spectra derivatives for all HyMap surveys shown in figure 1 below are freely available from the GSQ’s data portal.
Figure 1: Extent of hymap coverage from the Next Generation Mineral Mapping project. Image from Cudahy et al. (2008).
This blogpost and the accompanying code repository used raw hyperspectral data from the Dugald Block D survey. These raw data sets are divided into individual flight swaths with hyperspectral information stored as atmospheric corrected but non-georeferenced ENVI format reflectance grids, a portion of which is shown in figure 2 below. Thankfully, geometric corrections have been computed and are stored in geometry lookup tables for each reflectance grid. The notebook 01_apply_geocorrections_and_mosaic.ipynb reads geometry lookup tables and reflectance grids to output georeferenced ENVI format grids for each acquisition swath before using a simple GDAL procedure to merge them all into a single georeferenced reflectance mosaic of about 25GB.
Figure 2: Composite reflectance RGB for HyMap bands 9, 109 and 114 for a portion of an atmosphere corrected swath showing clear geometric effects related to the acquisition platform.
Vegetation
The notebook 02_vegetation_identification.ipynb runs a basic normalised difference vegetation index (NDVI) computation to identify vegetation in the newly created reflectance mosaic. The iterative use of native block windows of the mosaic, which in the case of ENVI grids are rows of pixels, makes this procedure memory efficient and can scale to much larger data sets. The final output of the notebook is a vegetation mask based on an entirely arbitrary NDVI cut-off of 0.55. There are better ways of identifying vegetation but they are beyond the scope of this demonstration.
Figure 3: Portion of the Dugald Bock D survey zoomed into Lake Julius showing NDVI (left) and the vegetation mask (right).
Convex Hull Continuum Removal
The notebook 03_convex_hull_computation.ipynb applies a convex hull continuum removal procedure to yield a new, continuum removed reflectance mosaic. Continuum removal is important as it attenuates the effect of variable illumination related to topography and allows for the comparison of absorption features from a common baseline. Figure 4 below presents a comparison between raw and continuum removed portions of spectral data sets around Lake Julius.
Figure 4. Comparison of raw (left) and continuum removed (right) spectral responses as RGB composite imagery for bands 32, 107 and 113. Note the attenuation of both topographic illumination and swath effects in the continuum removed image.
This is by far the slowest procedure in the code repository as the convex hull fitting and removal step had to be performed on a pixel-by-pixel basis inside a for loop. Effort was expended searching for a vectorised implementation of the convex hull fitting step but this bore no fruit. I tend to use a GPU-accelerated signal processing approach for continuum removal for this reason, but if any readers have suggestions to speed up the convex hull method they would be appreciated.
White Mica Abundance and Composition
On completion of the continuum removal procedure, a fairly basic white mica abundance and composition procedure was implemented in 04_white_mica_abundance.ipynb. This involved fitting a 2nd order polynomial to a window of the continuum removed spectrum where white mica minerals exhibit strong absorption. The magnitude and position of the absorption feature, proxies for abundance and composition, respectively, were approximated using the reflectance and wavelength position of the parabolic inflection point. In the cases where the fitted polynomial yielded an inflection point outside the spectral window in question, or was not U-shaped, the workflow recorded a null value. Figure 5 illustrates how the function is fitted to six random spectra, while figure 6 presents spatial representations of the outputs around Lake Julius.
Figure 5. Examples of the 2nd order polynomial white mica feature extraction workflow. If the fitted parabola has an inflection point with wavelength outside the white mica window then the workflow writes null values for the feature depth and wavelength position.
Figure 6. Zoomed in portions of the feature depth (left) and feature position (right) outputs for the white mica investigation around Lake Julius. White areas represent pixels that did not have absorption features in white mica region of the spectrum.
Conclusion
Manipulating, processing and interpreting relatively large hyperspectral data cubes does not necessarily require expensive proprietary software or overly sophisticated computing resources. This can be achieved through the use of a windowed approach to reading and writing data cubes, a method that can scale to much larger data cubes with many more spectral bands. From this point, more sophisticated machine learning and analysis methods can be applied to generate new, useful exploration products from freely available hyperspectral data sets.
References
Cudahy, T., Jones, M., Thomas, M., Laukamp, C., Caccetta, M., Hewson, R., … & Verrall, M. (2008). Next generation mineral mapping: Queensland airborne HyMap and satellite ASTER surveys 2006–2008. Commonwealth Scientific and Industrial Research Organisation Report, 364.
