Skip to main content
SpringerLink
Log in

Adaptive and stabilized real-time super-resolution control for UAV-assisted smart harbor surveillance platforms

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Nowadays, there are active research for deep learning applications to smart cities, e.g., smart factory, smart and micro grids, and smart logistics. Among them, for industrial smart harbor and logistics platforms, this paper proposes a novel two-stage algorithm for large-scale surveillance. For the purpose, this paper utilizes drones for flexible localization, and thus, the algorithm for scheduling between multiple drones and multiple multi-access edge computing (MEC) systems is proposed under the consideration of stability in this first-stage. After the scheduling, each drone transmits its own data to its associated MEC for enhancing the quality and then eventually the data will be used for surveillance. For improving the quality, super-resolution is used. In the second-stage algorithm, the self-adaptive super-resolution control is proposed for time-average performance maximization subject to stability, inspired by Lyapunov optimization. Based on data-intensive simulation results, it has been verified that the proposed algorithm achieves desired performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. binti Ahamad, N.B., Su, C., Zhaoxia, X., Vasquez, J.C., Guerrero, J.M., Liao, C.: Energy harvesting from harbor cranes with flywheel energy storage systems. IEEE Trans. Ind. Appl. 55(4), 3354–3364 (2019). https://doi.org/10.1109/TIA.2019.2910495

    Article  Google Scholar 

  2. Bethanabhotla, D., Caire, G., Neely, M.J.: Adaptive video streaming for wireless networks with multiple users and helpers. IEEE Trans. Commun. 63(1), 268–285 (2015). https://doi.org/10.1109/TCOMM.2014.2378774

    Article  Google Scholar 

  3. Capria, A., Giusti, E., Moscardini, C., Conti, M., Petri, D., Martorella, M., Berizzi, F.: Multifunction imaging passive radar for harbour protection and navigation safety. IEEE Aerosp. Electron. Syst. Mag. 32(2), 30–38 (2017). https://doi.org/10.1109/MAES.2017.160025

    Article  Google Scholar 

  4. Cheng, D., Meng, G., Xiang, S., Pan, C.: FusionNet: edge aware deep convolutional networks for semantic segmentation of remote sensing harbor images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(12), 5769–5783 (2017). https://doi.org/10.1109/JSTARS.2017.2747599

    Article  Google Scholar 

  5. Choi, M., Kim, J., Moon, J.: Wireless video caching and dynamic streaming under differentiated quality requirements. IEEE J. Sel. Areas Commun. 36(6), 1245–1257 (2018). https://doi.org/10.1109/JSAC.2018.2844980

    Article  Google Scholar 

  6. Jeong, S., Na, W., Kim, J., Cho, S.: Internet of things for smart manufacturing system: trust issues in resource allocation. IEEE Internet Things J. 5(6), 4418–4427 (2018). https://doi.org/10.1109/JIOT.2018.2814063

    Article  Google Scholar 

  7. Kim, D., Kwon, J., Kim, J.: Low-complexity online model selection with Lyapunov control for reward maximization in stabilized real-time deep learning platforms. In: Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 4363–4368 (2018)

  8. Kim, D., Kim, J., Kwon, J., Kim, T.: Depth-controllable very deep super-resolution network. In: Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2019). https://doi.org/10.1109/IJCNN.2019.8851874

  9. Kim, J.: Energy-efficient dynamic packet downloading for medical IoT platforms. IEEE Trans. Ind. Inform. 11(6), 1653–1659 (2015). https://doi.org/10.1109/TII.2015.2434773

    Article  Google Scholar 

  10. Kim, J., Meng, F., Chen, P., Egilmez, H.E., Bethanabhotla, D., Molisch, A.F., Neely, M.J., Caire, G., Ortega, A.: Adaptive video streaming for device-to-device mobile platforms. In: Proceedings of ACM International Conference on Mobile Computing and Networking (MobiCom), ACM, pp. 127–130 (2013)

  11. Kim, J., Caire, G., Molisch, A.F.: Quality-aware streaming and scheduling for device-to-device video delivery. IEEE/ACM Trans. Netw. 24(4), 2319–2331 (2016)

    Article  Google Scholar 

  12. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1646–1654 (2016). https://doi.org/10.1109/CVPR.2016.182

  13. Kim, J., Mo, Y.J., Lee, W., Nyang, D.: Dynamic security-level maximization for stabilized parallel deep learning architectures in surveillance applications. In: Proceedings of IEEE Symposium on Privacy-Aware Computing (PAC), pp. 192–193 (2017)

  14. Koo, J., Yi, J., Kim, J., Hoque, M.A., Choi, S.: REQUEST: seamless dynamic adaptive streaming over HTTP for multi-homed smartphone under resource constraints. In: Proceedings of ACM International Conference on Multimedia (MM), pp. 934–942 (2017)

  15. Koo, J., Yi, J., Kim, J., Hoque, M.A., Choi, S.: Seamless dynamic adaptive streaming in LTE/Wi-Fi integrated network under smartphone resource constraints. IEEE Trans. Mob. Comput. 18(7), 1647–1660 (2019)

    Article  Google Scholar 

  16. Moeller, S., Sridharan, A., Krishnamachari, B., Gnawali, O.: Routing without routes: the backpressure collection protocol. In: Proceedings of the 9th ACM International Conference on Information Processing in Sensor Networks (IPSN), pp. 279–290 (2010)

  17. Neely, M.J.: Energy optimal control for time-varying wireless networks. IEEE Trans. Inf. Theory 52(7), 2915–2934 (2006)

    Article  MathSciNet  Google Scholar 

  18. Neely, M.J.: Stochastic Network Optimization with Application to Communication and Queueing Systems. Synthesis Lectures on Communication Networks, Morgan & Claypool Publishers, San Rafael (2010)

    Book  Google Scholar 

  19. Neely, M.J.: Stock market trading via stochastic network optimization. In: Proceedings of IEEE Conference on Decision and Control (CDC), pp. 2777–2784 (2010)

  20. Neely, M.J.: Dynamic optimization and learning for renewal systems. IEEE Trans. Autom. Control 58(1), 32–46 (2013)

    Article  MathSciNet  Google Scholar 

  21. Neely, M.J., Saber, Tehrani, A., Dimakis, A.G.: Efficient algorithms for renewable energy allocation to delay tolerant consumers. In: Proceedings of IEEE International Conference on Smart Grid Communications, pp. 549–554 (2010)

  22. Park, L., Jeong, S., Kim, J., Cho, S.: Joint geometric unsupervised learning and truthful auction for local energy market. IEEE Trans. Ind. Electron. 66(2), 1499–1508 (2019). https://doi.org/10.1109/TIE.2018.2849979

    Article  Google Scholar 

  23. Shin, M., Kim, J., Levorato, M.: Auction-based charging scheduling with deep learning framework for multi-drone networks. IEEE Trans. Veh. Technol. 68(5), 4235–4248 (2019). https://doi.org/10.1109/TVT.2019.2903144

    Article  Google Scholar 

  24. Wang, W., Jost, T., Raulefs, R.: A semi-deterministic path loss model for in-harbor LoS and NLoS environment. IEEE Trans. Antennas Propag. 65(12), 7399–7404 (2017). https://doi.org/10.1109/TAP.2017.2765739

    Article  Google Scholar 

  25. Xiao, P., Tian, X., Liu, M., Liu, W.: Multipath smearing suppression for synthetic aperture radar images of harbor scenes. IEEE Access 7, 20150–20162 (2019). https://doi.org/10.1109/ACCESS.2019.2897779

    Article  Google Scholar 

  26. Zanella, A., Bui, N., Castellani, A., Vangelista, L., Zorzi, M.: Internet of things for smart cities. IEEE Internet Things J. 1(1), 22–32 (2014). https://doi.org/10.1109/JIOT.2014.2306328

    Article  Google Scholar 

  27. Zhuang, Y., Wang, P., Yang, Y., Shi, H., Chen, H., Bi, F.: Harbor water area extraction from pan-sharpened remotely sensed images based on the definition circle model. IEEE Geosci. Remote Sens. Lett. 14(10), 1690–1694 (2017). https://doi.org/10.1109/LGRS.2017.2728825

    Article  Google Scholar 

Download references

Acknowledgements

The research is supported by the Ministry of Health and Welfare (MHW) of Korea (HI19C0842).

Funding

This study was funded by National Research Foundation of Korea (Grant no. 2019M3E3A1084054).

Author information

Authors and Affiliations

  1. Hallym University, Engineering Building 1210, 1 Hallimdaehak-gil, Chuncheon, 24252, Korea

    Soyi Jung

  2. Korea University, Engineering Building 214, Anam-ro 145, Seongbuk-gu, Seoul, 02841, Korea

    Joongheon Kim

Authors
  1. Soyi Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joongheon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joongheon Kim.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jung, S., Kim, J. Adaptive and stabilized real-time super-resolution control for UAV-assisted smart harbor surveillance platforms. J Real-Time Image Proc 18, 1815–1825 (2021). https://doi.org/10.1007/s11554-021-01163-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-021-01163-2

Keywords

Search

Navigation