Computer Science and Information Systems 2023 Volume 20, Issue 4, Pages: 1289-1310
https://doi.org/10.2298/CSIS230315054Y
Full text (
593 KB)
Cited by
M2F2-RCNN: Multi-functional faster RCNN based on multi-scale feature fusion for region search in remote sensing images
Yin Shoulin (College of Information and Communication Engineering, Harbin Engineering University Harbin, China), yslin@hit.edu.cn
Wang Liguo (College of Information and Communications Engineering, Dalian Minzu University Dalian, China), wangliguo@hrbeu.edu.cn
Wang Qunming (College of Surveying and Geo-Informatics, Tongji University Shanghai, China), wqm@163.com
Ivanović Mirjana 
(Faculty of Sciences, University of Novi Sad Novi Sad, Serbia), mira@dmi.uns.ac.rs
Yang Jinghui (School of Information Engineering, China University of Geosciences Beijing, China), yang06081102@163.com
In order to realize fast and accurate search of sensitive regions in remote sensing images, we propose a multi-functional faster RCNN based on multi-scale feature fusion model for region search. The feature extraction network is based on ResNet50 and the dilated residual blocks are utilized for multi-layer and multi-scale feature fusion. We add a path aggregation network with a convolution block attention module (CBAM) attention mechanism in the backbone network to improve the efficiency of feature extraction. Then, the extracted feature map is processed, and RoIAlign is used to improve the pooling operation of regions of interest and it can improve the calculation speed. In the classification stage, an improved nonmaximum suppression is used to improve the classification accuracy of the sensitive region. Finally, we conduct cross validation experiments on Google Earth dataset and the DOTA dataset. Meanwhile, the comparison experiments with the state -of the- art methods also prove the high efficiency of the proposed method in region search ability.
Keywords: remote sensing images, region search, multi-functional faster RCNN, multi-scale feature fusion, convolution block attention module
Show references
Wang Z, Li P, Cui Y, et al. ”Automatic Detection of Individual Trees in Forests Based on Airborne LiDAR Data with a Tree Region-Based Convolutional Neural Network (RCNN),” Remote Sensing, 2023, 15(4): 1024.
Seetharaman K, Mahendran T. Leaf Disease Detection in Banana Plant using Gabor Extraction and Region-Based Convolution Neural Network (RCNN)[J]. Journal of The Institution of Engineers (India): Series A, 2022, 103(2): 501-507.
Y. Yuan, Z. Xu and G. Lu, ”SPEDCCNN: Spatial Pyramid-Oriented Encoder-Decoder Cascade Convolution Neural Network for Crop Disease Leaf Segmentation,” IEEE Access, vol. 9, pp. 14849-14866, 2021, doi: 10.1109/ACCESS.2021.3052769.
Yanmaz E. Joint or decoupled optimization: Multi-UAV path planning for search and rescue[J]. Ad Hoc Networks, 2023, 138: 103018.
Teng, L., Qiao, Y. BiSeNet-oriented context attention model for image semantic segmentation. Computer Science and Information Systems, vol. 19, no. 3, pp. 1409-1426. (2022), https://doi.org/10.2298/CSIS220321040T.
Uijlings J R R, Van De Sande K E A, Gevers T, et al. Selective search for object recognition[J]. International journal of computer vision, 2013, 104: 154-171.
Tang X, Xie X, Hao K, et al. A line-segment-based non-maximum suppression method for accurate object detection[J]. Knowledge-Based Systems, 2022, 251: 108885.
Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks[J]. Advances in neural information processing systems, 2015, 28.
Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[ C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 779-788.
Liu W, Anguelov D, Erhan D, et al. Ssd: Single shot multibox detector[C] Computer Vision CECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I 14. Springer International Publishing, 2016: 21-37.
Karim, Shahid, Geng Tong, Jinyang Li, Akeel Qadir, Umar Farooq, and Yiting Yu. ”Current Advances and Future Perspectives of Image Fusion: A Comprehensive Review.” Information Fusion, Vol. 90, pp.185-217, February 2023.
S. Yin, L.Wang, M. Shafiq, L. Teng, A. A. Laghari and M. F. Khan, ”G2Grad-CAMRL: An Object Detection and Interpretation Model Based on Gradient-weighted Class Activation Mapping and Reinforcement Learning in Remote Sensing Images,” in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023. doi: 10.1109/JSTARS.2023.3241405.
Tang C, Wu J, Hou Y, et al. A spectral and spatial approach of coarse-to-fine blurred image region detection[J]. IEEE Signal Processing Letters, 2016, 23(11): 1652-1656.
Wu D, Cao L, Zhou P, et al. Infrared Small-Target Detection Based on Radiation Characteristics with a Multimodal Feature Fusion Network[J]. Remote Sensing, 2022, 14(15): 3570.
Li Z, Wang H, Zhong H, et al. Self-attention module and FPN-based remote sensing image target detection[J]. Arabian Journal of Geosciences, 2021, 14: 1-18.
Cui Z, Leng J, Liu Y, et al. SKNet: Detecting rotated ships as keypoints in optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(10): 8826- 8840.
Pazhani A A J, Vasanthanayaki C. Object detection in satellite images by faster R-CNN incorporated with enhanced ROI pooling (FrRNet-ERoI) framework[J]. Earth Science Informatics, 2022, 15(1): 553-561.
Anuar M M, Halin A A, Perumal T, et al. Aerial imagery paddy seedlings inspection using deep learning[J]. Remote Sensing, 2022, 14(2): 274.
Gao, Y., Wu, H., Wu, X., Li, Z., Zhao, X.: Human Action Recognition Based on Skeleton Features. Computer Science and Information Systems, Vol. 14, No. 3, 537-550. (2017), https://doi.org/10.2298/CSIS220131067G
Chen L, Yang Y, Wang Z, et al. Underwater Target Detection Lightweight Algorithm Based on Multi-Scale Feature Fusion[J]. Journal of Marine Science and Engineering, 2023, 11(2): 320.
Zheng C, Wang L. Semantic segmentation of remote sensing imagery using object-based Markov random field model with regional penalties[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 8(5): 1924-1935.
Ma F, Gao F, Wang J, et al. A novel biologically-inspired target detection method based on saliency analysis for synthetic aperture radar (SAR) imagery[J]. Neurocomputing, 2020, 402: 66-79.
Wang Z, Xu N, Wang B, et al. Urban building extraction from high-resolution remote sensing imagery based on multi-scale recurrent conditional generative adversarial network[J]. GIScience & Remote Sensing, 2022, 59(1): 861-884.
Ni K, Zhao Y, Wu Y. SAR Image Segmentation Based on Super-Pixel and Kernel-Improved CV Model[C]//IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2022: 3476-3479.
Zhu Z, Yin H, Chai Y, et al. A novel multi-modality image fusion method based on image decomposition and sparse representation[J]. Information Sciences, 2018, 432: 516-529.
Liguo Wang, Yin Shoulin, Hashem Alyami, et al. ”A novel deep learning-based single shot multibox detector model for object detection in optical remote sensing images,” Geoscience Data Journal, (2022). https://doi.org/10.1002/gdj3.162
Karnyoto, A. S., Sun, C., Liu, B., Wang, X.: Transfer Learning and GRU-CRF Augmentation for Covid-19 Fake News Detection. Computer Science and Information Systems, Vol. 19, No. 2, 639-658. (2022), https://doi.org/10.2298/CSIS210501053K
Farhadi A, Redmon J. Yolov3: An incremental improvement[C]//Computer vision and pattern recognition. Berlin/Heidelberg, Germany: Springer, 2018, 1804: 1-6.
Luo S, Yu J, Xi Y, et al. Aircraft target detection in remote sensing images based on improved YOLOv5[J]. Ieee Access, 2022, 10: 5184-5192.
Zhang K, Shen H. Multi-stage feature enhancement pyramid network for detecting objects in optical remote sensing images[J]. Remote Sensing, 2022, 14(3): 579.
Ma J, Shao W, Ye H, et al. Arbitrary-oriented scene text detection via rotation proposals[J]. IEEE transactions on multimedia, 2018, 20(11): 3111-3122.
Wang K, Du S, Liu C, et al. Interior attention-aware network for infrared small target detection[ J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-13.
Singhal S, Passricha V, Sharma P, et al. Multi-level region-of-interest CNNs for end to end speech recognition[J]. Journal of Ambient Intelligence and Humanized Computing, 2019, 10: 4615-4624.
Theckedath D, Sedamkar R R. Detecting affect states using VGG16, ResNet50 and SEResNet50 networks[J]. SN Computer Science, 2020, 1: 1-7.
Sun M, Zhao H, Li J. Road crack detection network under noise based on feature pyramid structure with feature enhancement (road crack detection under noise)[J]. IET Image Processing, 2022, 16(3): 809-822.
Lin T Y, Doll P, Girshick R, et al. Feature pyramid networks for object detection[ C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 2117-2125.
Addis A A, Tian W, Acheampong K N, et al. Small-Scale and Occluded Pedestrian Detection Using Multi Mapping Feature Extraction Function and Modified Soft-NMS[J]. Computational Intelligence and Neuroscience: CIN, 2022, 2022.
Hou Y, Yang Q, Li L, et al. Detection and Recognition Algorithm of Arbitrary-Oriented Oil Replenishment Target in Remote Sensing Image[J]. Sensors, 2023, 23(2): 767.
Lv N, Zhang Z, Li C, et al. A hybrid-attention semantic segmentation network for remote sensing interpretation in land-use surveillance[J]. International Journal of Machine Learning and Cybernetics, 2023, 14(2): 395-406.
XiongW, Xiong Z, Cui Y. An explainable attention network for fine-grained ship classification using remote-sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-14.
Chen J, Li Z, Peng C, et al. UAV Image Stitching Based on Optimal Seam and Half-Projective Warp[J]. Remote Sensing, 2022, 14(5): 1068.