Abstract
Nowadays, the electromagnetic space is complex and variable, and the accurate identification of modulated signals is becoming increasingly difficult. And model quality can directly affect signal recognition. Therefore, this paper constructs a model performance evaluation system based on the recognition effect of modulated signals, and builds a hierarchical model of evaluation indexes from the classification performance, complexity performance, noise robustness and adversarial robustness of the model, to make a comprehensive and credible evaluation of the model quality from multiple dimensions. Through the experiment, we found that the complexity and classification performance of the model can affect the robustness of the model to a certain extent. The results of the evaluation show that the evaluation system can make a comprehensive and reasonable assessment of the quality of the model under the modulation-based signal recognition task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, Y., Liu, Y., Yang, C.: Modulation recognition with graph convolutional network. IEEE Wirel. Commun. Lett. 9(5), 624–627 (2020)
Tu, Y., Lin, Y., Hou, C., Mao, S.: Complex-valued networks for automatic modulation classification. IEEE Trans. Veh. Technol. 69(9), 10085–10089 (2020)
Wang, J., Qi, L., Han, Y.: Modulation recognition based on complex binarized neural network. In: 2022 9th International Conference on Dependable Systems and Their Applications (DSA), pp. 126–132. IEEE (2022)
Zhang, Z., Wang, C., Gan, C., Sun, S., Wang, M.: Automatic modulation classification using convolutional neural network with features fusion of SPWVD and BJD. IEEE Trans. Signal Inf. Process. Netw. 5(3), 469–478 (2019)
Ya, T., et al.: Large-scale real-world radio signal recognition with deep learning. Chin. J. Aeronaut. 35(9), 35–48 (2022)
O’Shea, T.J., Roy, T., Clancy, T.C.: Over-the-air deep learning based radio signal classification. IEEE J. Sel. Top. Signal Process. 12(1), 168–179 (2018)
Wang, J., Zha, H., Fu, J.: Evaluation of deep learning model in the field of electromagnetic signal recognition. In: IEEE INFOCOM 2022-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 1–6. IEEE (2022)
Bao, Z., Lin, Y., Zhang, S., Li, Z., Mao, S.: Threat of adversarial attacks on dl-based IoT device identification. IEEE Internet Things J. 9(11), 9012–9024 (2021)
Doan, V.-S., Huynh-The, T., Hua, C.-H., Pham, Q.-V., Kim, D.-S.: Learning constellation map with deep CNN for accurate modulation recognition. In: GLOBECOM 2020-2020 IEEE Global Communications Conference, pp. 1–6. IEEE (2020)
Zhao, H., Lin, Y., Gao, S., Yu, S.: Evaluating and improving adversarial attacks on DNN-based modulation recognition. In: GLOBECOM 2020-2020 IEEE Global Communications Conference, pp. 1–5. IEEE (2020)
Cai, Z., Wang, J., Ma, M.: The performance evaluation of big data-driven modulation classification in complex environment. IEEE Access 9, 26313–26322 (2021)
Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019)
Kurakin, A., Goodfellow, I., Bengio, S.: Adversarial machine learning at scale, arXiv preprint arXiv:1611.01236 (2016)
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks, arXiv preprint arXiv:1706.06083 (2017)
Dong, Y., et al.: Boosting adversarial attacks with momentum. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9185–9193 (2018)
O’Shea, T.J., Corgan, J., Clancy, T.C.: Convolutional radio modulation recognition networks. In: Jayne, C., Iliadis, L. (eds.) EANN 2016. CCIS, vol. 629, pp. 213–226. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44188-7_16
Xu, J., Luo, C., Parr, G., Luo, Y.: A spatiotemporal multi-channel learning framework for automatic modulation recognition. IEEE Wirel. Commun. Lett. 9(10), 1629–1632 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Yang, S., Wang, M., Lin, Y. (2024). Research on Model Evaluation Technology Based on Modulated Signal Identification. In: Li, J., Zhang, B., Ying, Y. (eds) 6GN for Future Wireless Networks. 6GN 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 553. Springer, Cham. https://doi.org/10.1007/978-3-031-53401-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-53401-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-53400-3
Online ISBN: 978-3-031-53401-0
eBook Packages: Computer ScienceComputer Science (R0)