Skip to main content
SpringerLink
Log in

Distributed Analysis Dictionary Learning Using a Diffusion Strategy

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

We consider the problem of distributed dictionary learning which aims to learn a global dictionary from data geographically distributed on nodes of a network. Existing works are based on sparse synthesis model while this paper is based on sparse analysis model. Two novel distributed analysis dictionary learning (ADL) algorithms are proposed by adapting the centralized ADL algorithms Analysis SimCO (ASimCO) and Incoherent Analysis SimCO (INASimCO) to distributed settings. In particular, local representation vectors and local dictionaries are introduced, and they can be updated independently on each node by distributing the sparse coding and dictionary update stages of ASimCO. A diffusion strategy is then applied to estimate a global dictionary from the local dictionaries by exchanging local information. Experimental results with synthetic data and for image denoising demonstrate that the proposed distributed ADL algorithms can obtain similar results as correpsonding centralized algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. All simulations were performed in Matlab 2014a with an Intel Core i7 CPU at 2.40 GHz and 12 GB memory.

References

  1. Aharon M, Elad M, Bruckstein A (2006) K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  Google Scholar 

  2. Cattivelli FS, Sayed AH (2010) Diffusion LMS strategies for distributed estimation. IEEE Trans Signal Process 58(3):1035–1048

    Article  MathSciNet  Google Scholar 

  3. Chen J, Towfic ZJ, Sayed AH (2015) Dictionary learning over distributed models. IEEE Trans Signal Process 63(4):1001–1016

    Article  MathSciNet  Google Scholar 

  4. Dai W, Xu T, Wang W (2012) Simultaneous codeword optimization (SimCO) for dictionary update and learning. IEEE Trans Signal Process 60(12):6340–6353

    Article  MathSciNet  Google Scholar 

  5. Daneshmand A, Sun Y, Scutari G, Facchinei F (2017) D2l: Decentralized dictionary learning over dynamic networks. In: 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 4084–4088

  6. Dong J, Wang W, Dai W, Plumbley MD, Han Z, Chambers J (2016) Analysis SimCO algorithms for sparse analysis model based dictionary learning. IEEE Trans Signal Process 64(2):417–431

    Article  MathSciNet  Google Scholar 

  7. Elad M, Milanfar P, Rubinstein R (2007) Analysis versus synthesis in signal priors. Inverse Prob 23(3):947–968

    Article  MathSciNet  Google Scholar 

  8. Engan K, Aase S, Hakon-Husoy J (1999) Method of optimal directions for frame design. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), vol 5, pp 2443–2446

  9. Hawe S, Kleinsteuber M, Diepold K (2013) Analysis operator learning and its application to image reconstruction. IEEE Trans Image Process 22(6):2138–2150

    Article  MathSciNet  Google Scholar 

  10. He K, Cao X, Shi Y, Nie D, Gao Y, Shen D (2019) Pelvic organ segmentation using distinctive curve guided fully convolutional networks. IEEE Trans Med Imaging 38(2):585–595

    Article  Google Scholar 

  11. Huang K, Wu Y, Wen H, Liu Y, Gui W (2020) Distributed dictionary learning for high-dimensional process monitoring. Control Eng Pract 98:104386

    Article  Google Scholar 

  12. Khishe M, Mosavi MR (2020) Chimp optimization algorithm. Expert Syst Appl 149:113338

    Article  Google Scholar 

  13. Li H, He X, Tao D, Tang Y, Wang R (2018) Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning. Pattern Recogn 79:130–146

    Article  Google Scholar 

  14. Li B, Rencker L, Dong J, Luo Y, Plumbley MD, Wang W (2021) Sparse analysis model based dictionary learning for signal declipping. IEEE J Sel Top Signal Process 15(1):25–36

    Article  Google Scholar 

  15. Liang J, Zhang M, Zeng X, Yu G (2014) Distributed dictionary learning for sparse representation in sensor networks. IEEE Trans Image Process 23(6):2528–2541

    Article  MathSciNet  Google Scholar 

  16. Nam S, Davies ME, Elad M, Gribonval R (2013) The cosparse analysis model and algorithms. Appl Comput Harm Anal 34(1):30–56

    Article  MathSciNet  Google Scholar 

  17. Pandey P, Rahmati M, Bajwa WU, Pompili D (2021) Real-time in-network image compression via distributed dictionary learning. IEEE Trans Mobile Comput, pp 1–14

  18. Raja H, Bajwa WU (2016) Cloud K-SVD: A collaborative dictionary learning algorithm for big, distributed data. IEEE Trans Signal Process 64(1):173–188

    Article  MathSciNet  Google Scholar 

  19. Rubinstein R, Peleg T, Elad M (2013) Analysis K-SVD: A dictionary-learning algorithm for the analysis sparse model. IEEE Trans Signal Process 61(3):661–677

    Article  MathSciNet  Google Scholar 

  20. Takahashi T, Tomioka R, Yamanishi K (2014) Discovering emerging topics in social streams via link-anomaly detection. IEEE Trans Knowl Data Eng 26(1):120–130

    Article  Google Scholar 

  21. Tu S, Sayed AH (2012) Diffusion strategies outperform consensus strategies for distributed estimation over adaptive networks. IEEE Trans Signal Process 60(12):6217–6234

    Article  MathSciNet  Google Scholar 

  22. Xiao L, Boyd S (2004) Fast linear iterations for distributed averaging. Syst Control Lett 53(1):65–78

    Article  MathSciNet  Google Scholar 

  23. Yaghoobi M, Nam S, Gribonval R, Davies M (2013) Constrained overcomplete analysis operator learning for cosparse signal modelling. IEEE Trans Signal Process 61(9):2341–2355

    Article  Google Scholar 

  24. Yu X, Ye X, Gao Q (2019) Pipeline image segmentation algorithm and heat loss calculation based on gene-regulated apoptosis mechanism. Int J Press Vessels Pip 172:329–336

    Article  Google Scholar 

  25. Zhang K, Li J, Wang H, Liu X, Gao X (2018) Learning local dictionaries and similarity structures for single image super-resolution. Signal Process 142:231–243

    Article  Google Scholar 

  26. Zheng Q, Tian X, Yang M, Wu Y, Su H (2020) PAC-bayesian framework based drop-path method for 2D discriminative convolutional network pruning. Multidimens Syst Signal Process 31:793–827

    Article  MathSciNet  Google Scholar 

  27. Zheng Q, Yang M, Tian X, Jiang N, Wang D (2020) A full stage data augmentation method in deep convolutional neural network for natural image classification. Discret Dyn Nat Soc 2020:1–11

    MATH  Google Scholar 

  28. Zheng Q, Zhao P, Li Y, Wang H, Yang Y (2021) Spectrum interference-based two-level data augmentation method in deep learning for automatic modulation classification. Neural Comput Appl 33:7723–7745

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China (61906087), the Natural Science Foundation of Jiangsu Province of China (BK20180692), the National Natural Science Foundation of China (61772237), and the Six Talent Climax Foundation of Jiangsu (XYDXX-030).

Author information

Authors and Affiliations

  1. College of Electrical Engineering and Control Science, Nanjing Tech University, Nanjing, Jiangsu, China

    Jing Dong & Liu Yang

  2. College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin, Heilongjiang, China

    Chang Liu

  3. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu, China

    Xiaoqing Luo

  4. College of Computer Science and Technology, Harbin Engineering University, Harbin, Heilongjiang, China

    Jian Guan

Authors
  1. Jing Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoqing Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jian Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Dong.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, J., Yang, L., Liu, C. et al. Distributed Analysis Dictionary Learning Using a Diffusion Strategy. Neural Process Lett 54, 2267–2281 (2022). https://doi.org/10.1007/s11063-021-10729-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10729-x

Keywords

Search

Navigation