Optical-enhanced Oil Tank Detection in High-resolution SAR Images

Ruixiang Zhang$^1$, Haowen Guo$^1$, Fang Xu$^1$, Wen Yang$^{1,*}$, Huai Yu$^1$, Haijian Zhang$^1$, Gui-Song Xia$^{2}$,
$^1$EIS SPL, Wuhan University, Wuhan, China
$^2$CAPTAIN, Wuhan University, Wuhan, China

Data and code can be found here!

Abstract

In recent years, object detection in high-resolution SAR images has made significant progress, especially after the introduction of deep learning. However, the dense oil tanks, which are compactly arranged targets in SAR images, are still challenging to recognize due to the unique imaging mechanism of SAR. Inspired by human learning from comparison, we propose a multi-stage framework for oil tank detection in SAR images using optical image enhancement. Specifically, in the training stage, we build the teacher-student network to transfer the knowledge between the two modalities, where the optical oil tank features are used to guide SAR oil tank features learning. While in the inference stage, the network detects oil tank targets using only SAR images as input. Besides, a pre-training stage before training is applied to further improve the network's ability for SAR feature extraction, which is realized by the proposed paired optical-SAR self-supervised learning. To verify the effectiveness of the proposed method, we performed experiments on our newly built SpaceNet6-OTD dataset. Extensive experiments demonstrate that the proposed method can effectively improve the performance of SAR image oil tank detection.

Introduction

Synthetic Aperture Radar~(SAR) sensors have the unique capability to penetrate clouds and collect during all-weather, day, and night conditions, which has been widely used in both civilian and military fields . High-resolution SAR data can provide more valuable information when weather and cloud cover can obstruct traditional optical sensors, which are suitable for detecting and monitoring some typical objects, such as buildings, ships, planes, vehicles, and oil tanks. In this work, we focus on detecting oil tanks in high-resolution SAR images.

However, detecting the oil tanks in SAR images is very challenging due to the unique imaging mechanism of SAR images. The oil tank targets in the SAR images are usually dense and compact with more overlaps and discrete scattering centers, which increases the difficulty of location in detection. Figure. 1 shows some samples of SAR images with oil tanks. As we can see, the heavy scattering interference between the dense oil tanks causes a blurred and cluttered appearance, which is hard to distinguish even the experts of SAR image analysis.

For the existing methods, the unsatisfactory detection accuracy is mainly caused by the indistinguishable appearance of the oil tanks in high-resolution SAR images. So naturally, how can we teach the network learning to recognize the oil tank in SAR images? As we all know, when humans encounter an illegible image of an unknown modality, we often learn to recognize it by comparing the corresponding image of a recognizable modality. Inspired by that, we propose introducing optical images more suitable for human vision to guide the network to learn the feature of oil tanks in SAR images.

Figure 1. (a) is the high-resolution SAR image with compact oil tanks. The SAR Intensity is visualized with three polarizations (HH, VV, HV) displayed through the Red, Green, and Blue color channels. (b) is the corresponding optical image. (c) is the detection result of Faster R-CNN. (d) is the detection result of our method. (The green boxes refer to the correct detection results, the yellow boxes refer to the false detection results, and the red boxes refer to the missing targets.

Figure 2. The proposed pipeline for detecting oil tanks in SAR images, uses optical-SAR self-supervised learning to pre-train a better feature extractor as the initialization in the training stage. First, the optical network guides the SAR network as the teacher. Then, the model detects the oil tanks during inference using only SAR images as input.

Based on the above, we design a multi-stage pipeline for detecting oil tanks in SAR images with the enhancement of optical images, as shown in Figure. 2. First, Optical-guided self-supervised learning is used in the pre-training stage to obtain the feature extractor suitable for SAR images. Then, in the training stage, a teacher-student network is constructed to make the SAR branch learn how to recognize oil tanks from the optical branch through knowledge distillation. Finally, the model detects the oil tanks using only SAR images as input during the inference stage. Our contribution can be summarized as follows: We propose a pipeline that uses optical images to guide SAR images to train a better oil tank detector in SAR images. Paired optical images and SAR images are used in the training stage, while only SAR images are used in the inference stage. We introduce paired optical-SAR images into the self-supervised learning to obtain a better SAR image feature extractor, improving the downstream oil tank detection task by 8-9 %. We construct an oil tank detection dataset, SpaceNet6-OTD, to explore the effectiveness of the proposed method. The dataset contains 820 paired SAR-Optical images with carefully labeled bounding boxes.

Experimental Results

A Comparison of Different Methods on SpaceNet6-OTD

Figure 3. Visualization of detection results using our proposed method compared with other state-of-the-art methods. The green boxes refer to the correct detection results, the yellow boxes refer to the false detection results, and the red boxes refer to the missing targets.

Acknowledgements We would like to thank the anonymous reviewers for their valuable comments and contributions. This project has received funding from the National Natural Science Foundation of China (NSFC) under Grant 61771351. In addition, the numerical calculations in this article have been done on the supercomputing system in the Supercomputing Center, Wuhan University.

References Spacenet 6: Multi-sensor all weather mapping dataset [paper] J. Shermeyer, D. Hogan, J. Brown, A. Van Etten, N. Weir, F. Pacifici, R. Hansch, A. Bastidas, S. Soenen, T. Bacastow et al. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020. Bootstrap your own latent-a new approach to self-supervised learning [paper] J.-B. Grill, F. Strub, F. Altch ́e, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar et al. Advances in Neural Information Processing Systems (NIPS), 2020. EO-augmented building segmentation for airborne sar imagery [paper] J. Kim, S. Shin, S. Kim, and Y. Kim. IEEE Geoscience and Remote Sensing Letters (GRSL), 2021. Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer [paper] N. Komodakis and S. Zagoruyko International Conference on Learning Representations (ICLR), 2017.