Skip to main content
SpringerLink
Log in

A Comparison of Single- and Multi-Objective Programming Approaches to Problems with Multiple Design Objectives

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

In this paper, we propose and compare single- and multi-objective programming (MOP) approaches to the language model (LM) adaptation that require the optimization of a number of competing objectives. In LM adaptation, an adapted LM is found so that it is as close as possible to two independently trained LMs. The LM adaptation approach developed in this paper is based on reformulating the training objective of a maximum a posteriori (MAP) method as an MOP problem. We extract the individual at least partially conflicting objective functions, which yields a problem with four objectives for a bigram LM: The first two objectives are concerned with the best fit to the adaptation data while the remaining two objectives are concerned with the best prior information obtained from a general domain corpus. Solving this problem in an iterative manner such that each objective is optimized one after another with constraints on the rest, we obtain a target LM that is a log-linear interpolation of the component LMs. The LM weights are found such that all the (at least partially conflicting) objectives are optimized simultaneously. We compare the performance of the SOP- and MOP-based solutions. Our experimental results demonstrate that the ICO method achieves a better balance among the design objectives. Furthermore, the ICO method gives an improved system performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7

Similar content being viewed by others

Notes

  1. The divergences from unigram models as well as bigram models should be considered since backing-off is used when an unknown n-gram is observed during the recognition (test) stage.

References

  1. Mitchell, T. M. (1997). Machine learning. New York: McGraw-Hill.

    MATH  Google Scholar 

  2. Liu, G. P., & Kadirkamanathan, V. (1995). Learning with multi objective criteria. In Proc. of IEE conference on artificial neural networks (pp. 53–58).

  3. Braga, A. P., Takahashi, R. H. C., Costa, M. A., & Teixeira, R. A. (2006). Multi-objective algorithms for neural networks learning. In Y. Jin, (Ed.), Multi-objective machine learning (pp. 151–171). Berlin: Springer.

    Chapter  Google Scholar 

  4. Suttorp, T., & Igel, C. (2006). Multi-objective optimization of support vector machines. In Y. Jin, (Ed.), Multi-objective machine learning (pp. 199–220). Berlin: Springer.

    Chapter  Google Scholar 

  5. Bellegarda, J. R. (2000). Exploiting latent semantic information in statistical language modeling. Proceedings of IEEE, 88, 1279–1296.

    Article  Google Scholar 

  6. Yaman, S., & Lee, C.-H. (2006). An iterative constrained optimization approach to classifier design. In Proc. of the IEEE conference on acoustics, speech, and signal processing. Toulouse, France.

  7. Yaman, S., Chien, J.-T., & Lee, C.-H. (2007). Structural Bayesian language modeling and adaptation. In Proc. Interspeech. Antwerp, Belgium.

  8. Miettinen, K. (1999). Nonlinear multiobjective optimization. Berlin: Springer.

    MATH  Google Scholar 

  9. Das, I., & Dennis, J. (1996). Closer look at drawbacks of minimizing weighted sums of objectives for pareto set generation in multicriteria optimization problems. Rice University, Houston, TX: Tech. Rep. 96–36, Department of Computational and Applied Mathematics.

  10. Bishop, C. M. (2007). Pattern recognition and machine learning. Berlin: Springer.

    Google Scholar 

  11. Nocedal, J., & Wright, S. J. (1999). Numerical optimization. Berlin: Springer.

    Book  MATH  Google Scholar 

  12. Paul, D. B., & Baker, J. M. (1992). The design for the wall street journal-based CSR corpus. In The proc. of internation conference of spoken language processing, Banff, Alberta, Canada, September.

  13. Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. Cambridge: MIT.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, 30332-0250, USA

    Sibel Yaman & Chin-Hui Lee

Authors
  1. Sibel Yaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chin-Hui Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sibel Yaman.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yaman, S., Lee, CH. A Comparison of Single- and Multi-Objective Programming Approaches to Problems with Multiple Design Objectives. J Sign Process Syst 61, 39–50 (2010). https://doi.org/10.1007/s11265-008-0295-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-008-0295-2

Keywords

Search

Navigation