Deep Learning | Remote Sensing
Remote Sensing: Deep Learning for Land Cover Classification of Satellite Imagery Using Python
A detailed explanation and Implementation of the 3D-CNN model for land cover classification of satellite imagery using Python.
There is no need to articulate the importance of satellite imagery analysis in today’s world. The applications of satellite imagery have been crucial to solving a lot of problems not only related to humanity but also the ecosystem. With help of the extensive research in Artificial Intelligence-driven applications, we are now capable to forecast and detect natural disasters like floods, wildfires, e.t.c in real-time. Precision Agriculture, Mineral Exploration, Urban Planning, Defense, and the list goes on.
Traditionally, Support Vector Machine (SVM), K-Nearest Neighbor Classifier (K-NNC), Logistic Regression, and Tree-based Algorithms are used for land cover classification in satellite imagery. The advancements of deep learning and computational power helped to develop effective deep learning models for land cover classification.
In Particularly, Convolutional Neural Networks(CNN) are widely used architecture (1D CNN, 2D CNN, and 3D CNN) is used to extract potential spectral and spatial features which are key to doing the land cover classification of the satellite imagery.
In this article, we are going to take a look at the brief explanation and implementation of 3D CNN for land cover classification of satellite Imagery.
Table of Contents
- Introduction
- Sundarbans Satellite Imagery
- Implementation of 3D CNN for Land Cover Classification
- Results
- Conclusion
- References
Let’s Get Started…
Introduction
Generally, the images that we encounter in our daily life contain 3 bands i.e., RGB (red, blue, and green bands). The satellite images contain more than three bands which contains a diverse set of information about any specific geographical location. With the help of more data in the form of bands, we can understand and analyze the data effectively. The below image shows a satellite image cube that contains n-rows(R), n-columns(C), and n-bands(B). The sentinel-2 satellite image consists of the coastal aerosol, Near Infra-Red(NIR), Short Wave Infra-Red(SWIR), and RGB bands that with a different wavelength and resolution.
To understand the satellite image, you can think of a book. Every page is a band that contains specific data.
The satellite imagery analysis can be divided into two types Spectral and Spatial. In this article, we are going to do spatial analysis with the help of 3D CNN.
Sundarbans Satellite Imagery
The Sundarbans is one of the largest mangrove areas in the world situated across the delta formed by the river Ganga and Bramhaputra. It is home to a wide range of wildlife species including endangered species.
The above Google Map shows the Sundarbans region. we are going to use a part of the Sundarbans satellite data which is acquired using the Sentinel-2 Satellite on 27 January 2020. The below table shows the general band information that includes wavelength and spectral resolution.
Let’s get started..,
Read Data
let’s read the 12 bands of the Sundarbans satellite image using rasterio and stack them into an n-dimensional array using numpy.stack() method. The resultant data after stacking has the shape (12, 954, 298). The ground truth of the satellite image is read using the loadmat method from the scipy.io package. The ground truth has 6 classes which include water, plants, trees, bare land, e.t.c.
Data Visualization
These Sundarbans data have multiple numbers of bands that contain the data ranging from visible to infrared. So it is hard to visualize the data for the naked eye. The help of an RGB Composite Image makes it easier to understand the data effectively.
To plot RGB composite images, Using red, green, and blue bands(bands 4, 3, and 2) we can visualize the composite image. Since Python uses a zero-based index system, so you need to subtract a value of 1 from each index. Therefore, the index for the red band is 3, green is 2, and blue is 1.
The method plot_rgb from EarthPy package is used to visualize the composite image of the satellite image.
Let’s visualize the ground truth using the plot_bands method from eathpy.plot package.
The below figure shows the composite image and the ground truth of the Sundarbans satellite data.
As we discussed, the data contains 12 bands. let’s visualize each band using the EarhPy package. The method plot_bands() takes the stack of the bands and plots along with custom titles which can be done by passing unique titles for each image as a list of titles using the title= parameter.
Implementation of 3D CNN for Land Cover Classification
The 3D CNN needs the three-dimensional data as an input, so we need to break down the satellite image into patches every patch will have a class. The class label of the patch has been defined as the class of the center pixel of the patch. Every patch has dimensions (W, W, B). Where W and B are window size and no. of bands. The below image depicts the patch from the satellite image.
Before extracting patches from the satellite image, I have reduced the dimensions of the data to 5 using the Principal Component Analysis technique. The pictorial representation of the patch extraction is shown below.
Let’s create three-dimensional patches of the Sundarbans satellite image by applying Principal Component Analysis(PCA) to the data. The below code is used to create functions for implementing PCA, creating 3D patches, and to split data into train and test data in the ratio of 70:30. After splitting the data the train and test labels are one-hot encoded using to_categorical() method from tensorflow.keras package.
CNN Model
Let’s build a three-dimensional CNN with multiple layers such as Convolution, Dropout, and Dense Layers. The below code is used to create a 3D-CNN for land cover classification using TensorFlow.
The 3D-CNN model has a total of 1,204,098 trainable parameters. The below figure shows the summary of the developed 3D-CNN model.
Training
For training the model, I have used Adam optimizer, categorical cross-entropy, accuracy as a metric, and a couple of callbacks for model optimization.
Adam is an optimization algorithm that can be used instead of the classical stochastic gradient descent procedure to update network weights iterative based on training data.
A brief explanation of callbacks that I have used to train the 3D-CNN.
EarlyStopping: One technique to reduce overfitting in neural networks is to use early stopping. Early stopping prevents overtraining of your model by terminating the training process if it’s not really learning anything. This is pretty flexible — you can control what metric to monitor, how much it needs to change to be considered “still learning”, and how many epochs in a row it can falter before the model stops training.
ModelCheckpoint: This callback will save your model as a checkpoint file (in hdf5 or h5format) to disk after each successful epoch. You can actually set the output file to be dynamically named based on the epoch. You can also write either the loss value or accuracy value as part of the log’s file name.
TensorBoard: This callback logs after every batch of training to monitor your metrics, graphs, histograms, images, e.t.c.
The below code is used to compile and train the model.
The Accuracy and Loss graph of the training is shown below. The X-Axis represents epochs and Y-Axis represents the Percentage.
Results
The CNN model after training has 95.60% accuracy, Let’s see the confusion matrix which is a tabular representation often used to describe the performance of a classification model (or “classifier”) on a set of test data for which the true values are known.
The classification report shows accuracy along with classwise precision, recall, f1-score, and Support. The output is shown below:
Finally, let’s visualize the classification map of the Sundarbans satellite. The below code is used to create the classification map. The method perdict is used to predict the labels of the entire data and reshaped into the original size and visualized using the method plot_bands .
The below image shows the RGB composite image, ground truth, and classification map of the Sundarbans satellite data.
Conclusion
This article shows the implementation and training of 3D-CNN for land cover classification of Sundarbans satellite imagery using python.
It took months of hard work and learning to present this article. A clap would be highly appreciated.
For more tutorials and information on remote sensing use the below GitHub repository.
