Skip to main content
SpringerLink
Log in

High-resolution time-frequency representation with generative adversarial networks

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Signal representation in time-frequency (TF) domain is valuable in many applications including radar imaging and inverse synthetic aperture radar. TF representation allows us to identify signal components or features in a mixed time and frequency plane. There are several well-known tools, such as Wigner–Ville Distribution (WVD), short-time Fourier transform and various other variants for such a purpose. The main requirement for a TF representation tool is to give a high-resolution view of the signal such that the signal components or features are identifiable. A commonly used method is the reassignment process which reduces the cross-terms by artificially moving smoothed WVD values from their actual location to the center of the gravity for that region. In this article, we propose a novel reassignment method using the conditional generative adversarial network (CGAN). We train a CGAN to perform the reassignment process. Through examples, it is shown that the method generates high-resolution TF representations which are better than the current reassignment methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Availability of data and materials

All of the material is owned by the authors and/or no permissions are required

References

  1. Wehner, D.R.: High-Resolution Radar, 2nd edn. Artech House, Boston (1994)

    Google Scholar 

  2. Wang, Y., Ling, H., Chen, V.C.: ISAR motion compensation via adaptive joint time-frequency technique. IEEE Trans. Aerosp. Electron. Syst. 34(4), 670–677 (1998)

    Article  Google Scholar 

  3. Thwe, K.Z., War, N.: Environmental sound classification based on time-frequency representation. In: 2017 18th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), pp. 251–255 (2017). https://doi.org/10.1109/SNPD.2017.8022729

  4. Chu, S., Narayanan, S., Kuo, C.-C.J.: Environmental sound recognition with time-frequency audio features. IEEE Trans. Audio Speech Lang. Process. 17(6), 1142–1158 (2009). https://doi.org/10.1109/TASL.2009.2017438

    Article  Google Scholar 

  5. Mitra, V., Franco, H.: Time-frequency convolutional networks for robust speech recognition. In: 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pp. 317–323 (2015). https://doi.org/10.1109/ASRU.2015.7404811

  6. Claasen, T.A.C.M., Mecklenbraiuker, W.F.G.: The Wigner distribution—a tool for time-frequency signal analysis; part III: relations with other time-frequency signal transformations. Philips J. Res. 35(6), 372–389 (1980)

    MathSciNet  MATH  Google Scholar 

  7. Hlawatsch, F., Boudreaux-Bartels, G.F.: Linear and quadratic time-frequency signal representations. IEEE Signal Process. Mag. 9(2), 21–67 (1992)

    Article  Google Scholar 

  8. Flandrin, P., Auger, F., Chassande-Mottin, E.: Time-frequency reassignment: from principles to algorithms. In: Papandreou-Suppappola, A. (ed.) Applications in Time-Frequency Signal Processing, Ch. 5, pp. 179–203. CRC Press, Boca Raton (2003)

    MATH  Google Scholar 

  9. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks (2014). https://doi.org/10.48550/ARXIV.1406.2661

  10. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks (2016). https://doi.org/10. 48550/ARXIV.1611.07004

  11. Cohen, L.: Time-frequency distributions—a review. Proc. IEEE 77(7), 941–981 (1989)

    Article  Google Scholar 

  12. Jones, D.L., Baraniuk, R.G.: An adaptive optimal-kernel time-frequency representation. IEEE Trans. Signal Process. 43(10), 2361–2371 (1995)

    Article  Google Scholar 

  13. Deprem, Z., Çetin, A.E.: Crossterm-free time-frequency distribution reconstruction via lifted projections. IEEE Trans. Aerosp. Electron. Syst. 51(1), 1–13 (2015)

    Article  Google Scholar 

  14. Deprem, Z., Çetin, A.E.: Kernel estimation for time-frequency distribution. In: 2015 23nd Signal Processing and Communications Applications Conference (SIU), pp. 1973–1976 (2015). https://doi.org/10.1109/SIU.2015.7130250

  15. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  16. Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 60–652 (2005). https://doi.org/10.1109/CVPR.2005.38

  17. Eigen, D., Fergus, R.: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture (2014). https://doi.org/10.48550/ARXIV.1411.4734

  18. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. Comment: Under Review as a Conference Paper at ICLR 2016 (2015). arXiv:1511.06434

  19. Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization (2016). https://doi.org/10.48550/ARXIV.1603.08511

  20. Mathieu, M., Couprie, C., LeCun, Y.: Deep multi-scale video prediction beyond mean square error. In: 4th International Conference on Learning Representations, ICLR 2016; Conference Date: 02-05-2016 Through 04-05-2016 (2016)

  21. Wang, X., Gupta, A.: Generative image modeling using style and structure adversarial networks (2016). https://doi.org/10.48550/ARXIV.1603.05631

  22. Mirza, M., Osindero, S.: Conditional generative adversarial nets (2014). https://doi.org/10.48550/ARXIV.1411.1784

  23. Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomedical image segmentation (2015). https://doi.org/10.48550/ARXIV.1505.04597

  24. Auger, F., Flandrin, P., Gonçalves, P., Lemoine, O.: Time-frequency toolbox for use with MATLAB (1996). http://tftb.nongnu.org

  25. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., Kudlur, M., Levenberg, J., Monga, R., Moore, S., Murray, D.G., Steiner, B., Tucker, P., Vasudevan, V., Warden, P., Wicke, M., Yu, Y., Zheng, X.: Tensorflow: a system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 265–283 (2016). https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf

  26. Pearson, K.: Notes on regression and inheritance in the case of two parents. Proc. R. Soc. Lond. 58, 240–242 (1895)

    Article  Google Scholar 

  27. Baraniuk, R.G., Flandrin, P., Janssen, A.J.E.M., Michel, O.: Measuring time-frequency information content using the Rényi entropies. IEEE Trans. Inf. Theory 47(4), 1391–1409 (2001)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

Not applicable

Funding

The authors have no funding for this work.

Author information

Authors and Affiliations

  1. EEE Department, UTAA, 6004, Etimesgut, Ankara, Turkey

    Zeynel Deprem

  2. ECE Department, UIC, Taylor St., Chicago, IL, 60607, USA

    A. Enis Çetin

Authors
  1. Zeynel Deprem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Enis Çetin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZD wrote the main manuscript. All the authors reviewed the manuscript.

Corresponding author

Correspondence to Zeynel Deprem.

Ethics declarations

Conflict of interest

The authors have no competing interests as defined by Springer, or other interests that might be perceived to influence the results and/or discussion reported in this paper.

Consent to participate

The authors accept the ethical responsibilities and have the consent to participate.

Consent for publication

Not applicable.

Ethics approval

The results/data/figures in this manuscript have not been published elsewhere, nor are they under consideration by another publisher.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deprem, Z., Çetin, A.E. High-resolution time-frequency representation with generative adversarial networks. SIViP 17, 849–854 (2023). https://doi.org/10.1007/s11760-022-02297-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02297-x

Keywords

Search

Navigation