Abstract
Least-squares problems occur often in practice, for example, when a parametrized model is used to describe a behavior of a chemical, physical or an economic application. In this paper, we describe a method for solving least-squares problems that are given as a large system of equations. The solution combines the commonly used methods with algorithmic differentiation and shared-memory multiprocessing. The system of equations contains model functions that are independent from each other. This independence enables the usage of a multiprocessing approach. With help of algorithmic differentiation by source transformation, we obtain the derivative code of the residual function. The advantage of using source transformation is that we can transform the OpenMP pragmas of the input code into corresponding pendants in the derivative code. This is, in particular in the adjoint case, not a straightforward approach. We show the scaling properties of the derivative code and of the optimization process.
Chapter PDF
References
OpenMP Architecture Review Board. OpenMP Application Program Interface. Specification (2011)
IEEE. IEEE: Threads Extension for Portable Operating Systems (Draft 6). Specification (1992)
Nocedal, J., Wright, S.: Numerical Optimization, 2nd edn. Springer, New York (2006)
Griewank, A., Walter, A.: Evaluating Derivatives. Principles and Techniques of Algorithmic Differentiation, 2nd edn. SIAM, Philadelphia (2008)
Madsen, K., Nielsen, B., Tingleff, O.: Methods for Non-Linear Least-Squares Problems (2004)
Cao, J., Novstrup, K., Goyal, A., Midkiff, S., Caruthers, J.: A Parallel Levenberg-Marquardt Algorithm. In: Proceedings of the 23rd International Conference on Supercomputing, ICS 2009, pp. 450–459. ACM, New York (2009)
Bücker, M., Lang, B., an Mey, D., Bischof, C.: Bringing together automatic differentiation and OpenMP. In: ICS 2001: Proceedings of the 15th International Conference on Supercomputing, pp. 246–251. ACM, New York (2001)
Bücker, M., Lang, B., Rasch, A., Bischof, C., an Mey, D.: Explicit Loop Scheduling in OpenMP for Parallel Automatic Differentiation. In: Annual International Symposium on High Performance Computing Systems and Applications, p. 121 (2002)
Bücker, M., Rasch, A., Wolf, A.: A class of OpenMP applications involving nested parallelism. In: SAC 2004: Proceedings of the 2004 ACM Symposium on Applied Computing, pp. 220–224. ACM, New York (2004)
Kowarz, A., Walther, A.: Parallel Derivative Computation using ADOL-C. In: Nagel, W.E., Hoffmann, R., Koch, A. (eds.) 9th Workshop on Parallel Systems and Algorithms (PASA) held at the 21st Conference on the Architecture of Computing Systems (ARCS), Dresden, Germany, February 26. LNI, vol. 124, pp. 83–92. GI (2008)
Bischof, C., Guertler, N., Kowarz, A., Walther, A.: Parallel reverse mode automatic differentiation for OpenMP programs with ADOL-C, pp. 163–173. Springer, Berlin (2008)
Dahmen, W., Reusken, A.: Numerik für Ingenieure und Naturwissenschaftler. Springer-Lehrbuch. Springer (2008)
Naumann, U.: The Art of Differentiating Computer Programs. Software, Environments and Tools. Society for Industrial and Applied Mathematics (2012)
Hannemann, R., Tillack, J., Schmitz, M., Förster, M., Wyes, J., Nöh, K., von Lieres, E., Naumann, E., Wiechert, W., Marquardt, W.: First- and Second-Order Parameter Sensitivities of a Metabolically and Isotopically Non-Stationary Biochemical Network Model. In: Otter, M., Zimmer, D. (eds.) Proceedings of the 9th International Modelica Conference, pp. 641–648. Modelica Association (2012)
Schanen, M., Förster, M., Gendler, B., Naumann, U.: Compiler-based Differentiation of Higher-Order Numerical Simulation Codes using Interprocedural Checkpointing. International Journal on Advances in Software 5(1&2), 27–35 (2012)
Förster, M., Naumann, U., Utke, J.: Toward Adjoint OpenMP. Technical Report AIB-2011-13, RWTH Aachen (July 2011)
Ben-Ari, M.: Principles of Concurrent and Distributed Programming. Prentice-Hall International Series in Computer Science. Addison-Wesley (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Förster, M., Naumann, U. (2013). Solving a Least-Squares Problem with Algorithmic Differentiation and OpenMP. In: Wolf, F., Mohr, B., an Mey, D. (eds) Euro-Par 2013 Parallel Processing. Euro-Par 2013. Lecture Notes in Computer Science, vol 8097. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40047-6_76
Download citation
DOI: https://doi.org/10.1007/978-3-642-40047-6_76
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40046-9
Online ISBN: 978-3-642-40047-6
eBook Packages: Computer ScienceComputer Science (R0)