Skip to main content
Springer Nature Link
Log in

Missing Data Imputation for Time-Frequency Representations of Audio Signals

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

With the recent attention towards audio processing in the time-frequency domain we increasingly encounter the problem of missing data within that representation. In this paper we present an approach that allows us to recover missing values in the time-frequency domain of audio signals. The presented approach is able to deal with real-world polyphonic signals by operating seamlessly even in the presence of complex acoustic mixtures. We demonstrate that this approach outperforms generic missing data approaches, and we present a variety of situations that highlight its utility.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7

Similar content being viewed by others

Notes

  1. To be precise Eqs. 6 and 7 must actually be specified in terms of \(\mathcal{C}^{-1}N^o_t+1\); however, given the assumption in Eqs. 1 and 7, which is the primary equation of interest remains valid.

References

  1. Raj, B. (2000). Reconstruction of incomplete spectrograms for robust speech recognition. Ph.D. Dissertation, Carnegie Mellon University.

  2. Roweis, S. T. (2000). One microphone source separation (pp. 793–799). NIPS.

  3. Brand, M. E. (2002). Incremental singular value decomposition of uncertain data with missing values, European conference on computer vision (ECCV) (Vol. 2350, pp. 707–720).

  4. Reyes-Gomez, M. J., Jojic, N., & Ellis, D. P. W. (2004). Detailed graphical models for source separation and missing data interpolation in audio. Utah: Snowbird Learning Workshop Snowbird.

    Google Scholar 

  5. Le Roux, J, Kameoka, H., Ono, N., de Cheveigné, A., & Sagayama, S. (2008). Computational auditory induction by missing-data non-negative matrix factorization. Brisbane, Australia: SAPA.

    Google Scholar 

  6. Shashanka, M., Raj, B., & Smaragdis, P. (2000). Sparse overcomplete latent variable decomposition of counts data. NIPS.

  7. Smith, J. O. (2007). Spectral audio signal processing. March 2007 draft. http://ccrma.stanford.edu/~jos/sasp/. Accessed June 2008.

  8. David, M. H., Little, R. J. A., Samuhel, M. E., & Triest, R. K. (1983). Imputation methods based on the propensity to respond. In Proceedings of the business and economics section, American statistical association.

  9. Quinlan, J. R. (1989). Unknown attribute values in induction. In Proc. of the sixth international conference on machine learning.

  10. Ghaharamani, Z., & Jordan, M. I. (1994). Learning from incomplete data. Technical report AI Memo 1509. Artificial Intelligence Laboratory, MIT.

  11. Raj, B., Seltzer, M. L., & Stern, R. M. (2004). Reconstruction of missing features for robust speech recognition. Speech Communication Journal, 43(4), 275–296.

    Article  Google Scholar 

  12. Hastie, T., Tibshirani, R., Sherlock, G., Eisen, M., Brown, P., & Botstein, D. (1999). Imputing missing data for gene expression arrays. Technical report. Stanford Statistics Department.

  13. Hofmann, T. (2000). Learning the similarity of documents: An information-geometric approach to document retrieval and categorization. In Advances in neural information processing systems (Vol. 12, pp. 914–920). Cambridge: MIT Press.

    Google Scholar 

  14. Hofmann, T., & Puzicha, J. (1998). Unsupervised learning from dyadic data. TR 98-042. Berkeley: ICSI.

    Google Scholar 

  15. Hazewinkel, M. Encyclopedia of mathematics. http://eom.springer.de/.

  16. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, B, 39, 1–38.

    MathSciNet  MATH  Google Scholar 

  17. Griffin, D. W., & Lim, J. S. (1984). Signal reconstruction from short-time Fourier transform magnitude. IEEE Transactions of Acoustics, Speech, and Signal Processing, 32(2), 236–243.

    Article  Google Scholar 

  18. Bouvrie, J., & Ezzat, T. (2006). An incremental algorithm for signal reconstruction from short-time Fourier transform magnitude, in interspeech. USA: Pittsburgh.

    Google Scholar 

  19. Gould, G. (1994). Bach: The two and three part inventions—the Glenn Gould edition, by SONY classics. ASIN B000GF2YZ8.

Download references

Author information

Authors and Affiliations

  1. Adobe Systems Inc., Cambridge, MA, USA

    Paris Smaragdis

  2. Carnegie Mellon University, Pittsburgh, PA, USA

    Bhiksha Raj

  3. United Technologies Research Center, 411 Silver Lane, East Hartford, CT, 06108, USA

    Madhusudana Shashanka

Authors
  1. Paris Smaragdis
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Bhiksha Raj
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Madhusudana Shashanka
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Paris Smaragdis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Smaragdis, P., Raj, B. & Shashanka, M. Missing Data Imputation for Time-Frequency Representations of Audio Signals. J Sign Process Syst 65, 361–370 (2011). https://doi.org/10.1007/s11265-010-0512-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-010-0512-7

Keywords