Skip to main content

Advertisement

SpringerLink
Log in

Design and Implementation of an FPGA-Based DNN Architecture for Real-time Outlier Detection

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Deep neural networks (DNNs) have recently become the standard tool for solving practical problems in various applications, including timely data analysis and near real-time accurate decision-making. DNNs have proven effective in outlier detection, one of sensor networks’ primary motivating data analysis applications. Despite the great potential of deep neural networks, their computational resource requirements create a vast gap when it comes to the fast processing time required in real-time applications using low-power, low-cost edge devices. Special care must be taken into account when designing DNNs computational units. This work proposes an FPGA-based Deep Neural Network (DNN) architecture for real-time outlier detection in time series data. The proposed architecture integrates a fine-tuned Autoencoder network and a Long short-term memory (LSTM) network to predict and detect outliers in real-time. The hardware accelerator of the integrated networks combines serial-parallel computation with matrix algebra concepts to reduce computational complexity and enhance the throughput. Experimental results on the resource-constrained Xilinx PYNQ-Z1 board using an open-source sensor network dataset show that the proposed architecture can efficiently analyze and detect outliers in real-time. The implemented design achieves 0.22 ms average latency and 1GOPS throughput. The proposed design’s low latency and 94mW power consumption make it suitable for resource-constrained edge platforms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17

Similar content being viewed by others

Data Availability

The dataset analysed during the current study are available in the Zenodo repository. https://urldefense.com/v3/__https://doi.org/10.5281/zenodo.2654726__;!!NLFGqXoFfo8MMQ!qpTJPWL5LhIFyH93DgHuI-vIX-0Yqoczd9YfMmyBAXohn9PMvJTCc4y5xpWxvQz40VaGq8guaRzcFnwHwuxDwHbIirkJ8xcVHzI$

References

  1. Oliveira, L. M., & Rodrigues, J. J. (2011). Wireless sensor networks: a survey on environmental monitoring. Journal of Communications, 6(2), 143–151.

    Article  Google Scholar 

  2. Hua, G., Li, Y., & Yan, X. (2011). Research on the wireless sensor networks applied in the battlefield situation awareness system. International Conference ECWAC, 2011(April), 16–17.

    Google Scholar 

  3. Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), 15:1–15:58.

  4. Rajasegarar, S., Leckie, C., & Palaniswami, M. (2008). Anomaly detection in wireless sensor networks. IEEE Wireless Communications, 15(4).

  5. O’Reilly, C., Gluhak, A., Imran, M. A., & Rajasegarar, S. (2014, Third Quarter). Anomaly detection in wireless sensor networks in a non-stationary environment. IEEE Communications Surveys and Tutorials, 16(3), 1413–1432. https://doi.org/10.1109/SURV.2013.112813.00168

  6. Xie, M., Hu, J., Han, S., & Chen, H.-H. (2013). Scalable hypergrid k-NN based online anomaly detection in wireless sensor networks. IEEE Transactions on Parallel and Distributed Systems, 24(8), 1661–1670.

    Article  Google Scholar 

  7. Chen, P.-Y., Yang, S., & McCann, J. A. (2015). Distributed real-time anomaly detection in networked industrial sensing systems. IEEE Transactions on Industrial Electronics, 62(6), 3832–3842.

    Article  Google Scholar 

  8. Zhang, Y., Meratnia, N., & Havinga, P. J. (2013). Distributed online outlier detection in wireless sensor networks using ellipsoidal support vector machine. Ad hoc networks, 11(3), 1062–1074.

    Article  Google Scholar 

  9. Barrenetxea, G. (2019). Sensorscope Data [Data set]. Zenodo. https://doi.org/10.5281/zenodo.2654726

  10. PYNQ-Z1: Python Productivity for Zynq-7000 ARM/FPGA SoC. Digilent [online]. Retrieved June 6, 2013, from https://urldefense.com/v3/__https://store.digilentinc.com/pynq-z1-python-productivity-for-zynq-7000-arm-fpgasoc/__;!!NLFGqXoFfo8MMQ!qpTJPWL5LhIFyH93DgHuI-vIX-0Yqoczd9YfMmyBAXohn9PMvJTCc4y5xpWxvQz40VaGq8guaRzcFnwHwuxDwHbIirkJZkg6D6s$

  11. Vivado Design Suite. Xilinx [online]. Retrieved June 6, 2013, from https://urldefense.com/v3/__https://www.xilinx.com/products/design-tools/vivado.html__;!!NLFGqXoFfo8MMQ!qpTJPWL5LhIFyH93DgHuI-vIX-0Yqoczd9YfMmyBAXohn9PMvJTCc4y5xpWxvQz40VaGq8guaRzcFnwHwuxDwHbIirkJWiUTbuQ$

  12. Chollet, F., et al. (2015). Keras. Retrieved June 6, 2013, from https://urldefense.com/v3/__https://keras.io__;!!NLFGqXoFfo8MMQ!qpTJPWL5LhIFyH93DgHuI-vIX-0Yqoczd9YfMmyBAXohn9PMvJTCc4y5xpWxvQz40VaGq8guaRzcFnwHwuxDwHbIirkJLJLb30E$

  13. Mohamed, N. A., & Cavallaro, J. R. (2021). Real-time FPGA-based outlier detection using autoencoder and LSTM. 2021 55th Asilomar Conference on Signals, Systems, and Computers (pp. 1195–1199). https://doi.org/10.1109/IEEECONF53345.2021.9723300

  14. Guo, W., Wang, J., & Wanga, S. (2019). Deep multimodal representation learning: A survey. IEEE Access, 7, 63373–63394.

    Article  Google Scholar 

  15. Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H et al (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10.3115/v1/d14-1179

  16. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  17. Han, S., Kang, J., Mao, H., Hu, Y., Li, X., Li, Y., Xie, D., Luo, H., Yao, S., Wang, Y., et al. (2017). ESE: Efficient speech recognition engine with sparse LSTM on FPGA. In Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (pp. 75–84). ACM.

  18. Chang, A. X. M., & Culurciello, E. (2017). Hardware accelerators for recurrent neural networks on FPGA. In 2017 IEEE International Symposium on Circuits and Systems (ISCAS), ser. ISCAS ’17.

  19. Guan, Y., Liang, H., Xu, N., Wang, W., Shi, S., Chen, X., Sun, G., Zhang, W., & Cong, J. (April 2017). FP-DNN: An automated framework for mapping deep neural networks onto FPGAs with RTL-HLS hybrid templates. In 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) (pp. 152-159).

  20. Shin D., Lee J., Lee J., & Yoo, H. (Feb 2017). 14.2 DNPU: An 8.1TOPS/W reconfigurable CNN-RNN processor for general-purpose deep neural networks. In 2017 IEEE International Solid-State Circuits Conference (ISSCC) (pp. 240–241).

  21. Han, S., Mao, H., & Dally, W. J. (2015). Deep Compression: Compressing deep neural networks with pruning, trained quantization and Huffman Coding, CoRR, vol. abs/1510.00149. [Online]. Preprint retrieved from http://arxiv.org/abs/1510.00149

  22. Que, Z., Liu, Y., Guo, C., Niu, X., Zhu, Y., & Luk, W. (2019). Real-time anomaly detection for flight testing using autoencoder and LSTM. International Conference on Field-Programmable Technology (ICFPT), 2019, 379–382. https://doi.org/10.1109/ICFPT47387.2019.00072

    Article  Google Scholar 

Download references

Funding

This work was supported in part by the US NSF under grant CNS-2016727.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Rice University, 6100 Main St, Houston, 77005, TX, USA

    Nadya Mohamed & Joseph Cavallaro

Authors
  1. Nadya Mohamed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joseph Cavallaro
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Both authors contributed to the study conception and design.

Corresponding author

Correspondence to Nadya Mohamed.

Ethics declarations

Conflicts of Interest

The authors have no conflicts of interest to declare relevant to this article content.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mohamed, N., Cavallaro, J. Design and Implementation of an FPGA-Based DNN Architecture for Real-time Outlier Detection. J Sign Process Syst 95, 845–861 (2023). https://doi.org/10.1007/s11265-023-01835-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-023-01835-1

Keywords

Search

Navigation