The Cloud-Aerosol LIDAR Infrared Pathfinder Satellite Observations (CALIPSO) project, launched collaboratively by NASA and CNES in 2006, seeks to improve understanding of cloud/aerosol systems in the Earth’s atmosphere via measurements from a continuously-orbiting satellite. One objective of the project is to classify regions of the atmosphere by dominant feature type, differentiating between cloud, aerosol, and clear air. Combining readings from onboard instruments, including the CALIOP LIDAR sensor, external weather data, and known atmospheric conditions, the CALIOP operational scene classification algorithms (COSCA) were designed to perform such classifications. An open question is whether machine learning (ML) can replicate or surpass the performance of this tailored solution.

This research seeks to develop an alternative procedure for performing scene classification of CALIPSO satellite data using fewer assumptions than the current operational algorithms. The focus of the methodology will be applying supervised machine learning techniques that can recognize patterns and distinctions that current methods may ignore.

The CALIPSO data products are routinely published in 5 km, 1 km, and 333 m horizontal resolution versions maintained in cloud, aerosol, and combined feature datasets. These datasets undergo processing from raw EOS Level 0 instrument data to the fully geolocated, classified, and cross-referenced Level 4 versions. A highlight of these datasets is the Vertical Feature Mask (VFM), which records the COSCA results in a condensed bit-encoded format, including feature type, subtype, ice/water phase, and quality assurance and confidence metrics (see Fig. 1). The data products also expose the accompanying geospatial instrument data; this most notably includes the parallel and perpendicular components of the 532 nm LIDAR backscatter and the total backscatter of the 1064 nm channel instrument. Correlating satellite instrument data with available weather sources also provides post-processed data products with temperature, humidity, and pressure estimates. By examining this data together as a multichannel image, we can conduct learning with the VFM classifications as the target classes, using either a primary type or subtype scope.

Previous authors' (Zeng et al., etc.) studies of the subject have yielded promising results, but have yet to attain an ideal measure of agreement with the operational algorithms for all feature types and profile scenarios and have limited scope of study. The classification problem presents several challenges, including a high degree of class frequency imbalance, variations in signal strength and signal-to-noise ratio, and missing or highly-uncertain classifications resulting from anomalous or attenuated signals.

Our initial research builds upon prior research and continues to seek new approaches towards the goal of achieving an accurate and general model. A large corpus of randomly-sampled 333m CALIPO data was collected from the ICARE Data and Services Centre repository to serve as the model input. An additional dataset focusing on the first quarter of 2020 was also collected to perform time-constrained experimentation and analysis. These samples provide the highest-resolution instrument and VFM data available.

Our current strategies can be categorized into three types: (1) 2D images, (2) 1D single-shot LIDAR signals, and (3) feature-based samples. (1) combines adjacent CALIOP profiles into multichannel 2D images. This strategy has the advantage of being able to leverage traditional convolutional methods and detect texture information that may not be caught by the COSCA or other approaches. However, this information has been shown to sometimes be irrelevant or even detrimental. (2) provides a fundamentally simpler approach by considering each individual LIDAR profile as input. While this does not allow for two-dimensional texture information to be utilized, there are still many convolutional methods that can be adapted into the 1D case. (3) operates on pre-located features detected by the CALIPSO feature-finding algorithm, meaning the problem can be reduced from a per-pixel classification problem into a simple single-class problem. It also mimics the input space of the COSCA. However, it relies on the a priori spatial separation of individual features and thus may not be as satisfactory as an entirely ML-based segmentation solution.

Initial experiments began with the 2D case and applied U-Net-style networks, which are often more capable than the SegNet-style algorithms used in prior research. U-Net architectures provide fully-convolutional segmentation via a spatially-contracting stage followed by an up-sampling stage while maintaining connections between layers of like dimensions. This method achieved high accuracy in differentiating between a subset of classes, but still underperformed with many key classifications. Next, the feature-based approach was used. This provided only a marginal improvement to accuracy while incurring the disadvantages described above. These methods may prove tenable if a sufficient edge-detecting preprocessing step is introduced to the classification pipeline, but this idea has not yet been explored. Finally, the 1D approach was used. After expanding the input dimensions to include altitudes outside the VFM and applying a 1D U-Net-style architecture, a substantial improvement was made. Yet there remain some classes with unsatisfactory agreement rates.

The ongoing seeks to combine the approaches above with recurrent network models to take advantage of the temporal nature of the LIDAR signal while still exploiting the textural information of the spatial data. Since feature types often appear naturally in localized clusters, we believe this approach will lead to improved results. We are also experimenting with more robust loss functions and accuracy metrics that better capture the true classification rates without penalizing invalid or blatantly unconfident ground truths. Combined, we hope to further improve our classifier to produce a highly-accurate and general tool for analyzing atmospheric phenomena.