Abstract
We present an approximate algorithm for reconstructing internals of multi-layer perceptrons from membership queries. The key component of the algorithm is a procedure for reconstructing weights of a single linear threshold unit. We prove that the approximation error, measured as the distance between the original and the reconstructed weights, is dropping exponentially fast with the number of queries. The procedure is combined with a labelling strategy that involves solving multiple Linear Programming problems. This combination results in an algorithm that extracts internals of multi-layer per ceptrons: the number of units in the first hidden layer, their weights, and a boolean funct ion that is computed by the remaining nodes. In practice, networks that compute boolean combinations of 10–15 hyperplanes can be reconst ructe d in several minutes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
D. Angluin. Queries revisited. In T. Z. N. Abe, R. Khardon, editor, Algorithmic Learning Theory, 12th International Conference, ALT 2001, Washington, DC, USA, November 25-28, 2001, 12–31. Springer, 2001.
S. Argamon-Engelson and I. Dagan. Committee-based sample selection for probabilistic classifiers. Journal of Artificial Intelligence Research, 11:335–360, 1999.
E. Baum. On learning a union of half spaces. Journal of Complexity, 6(1):67–101, 1990.
E. Baum. Neural net algorithms that learn in polynomial time from examples and queries. IEEE Transactions on Neural Networks, 2(1):5–19, 1991.
C. Bishop. Neural Newtorks for Pattern Recognition. Oxford University Press, 1995.
R. Bramley and B. Winnicka. Solving linear inequalities in a least squares sense. SIAM Journal on Scientific Computing, 17(1):275–286, 1996.
P. Goldberg and S. Kwek. The precision of query points as a resource for learning convex polytopes with membership queries. In Proc. 13th Annu. Conference on Comput. Learning Theory, pages 225–235. Morgan Kaufmann, San Francisco, 2000.
M. Hasenjager and H. Ritter. Active learning in neural networks.
V. Shevchenko. On deciphering a threshold function of many-values logic. Gorkü State University, pages 155–166, 1987.
G. Strang. Linear algebra and its applications. Third Edition. Harcourt Brace Jovanovich, Publishers, 1988.
L.G. Valiant. A theory of the learnable. Communications ACM, vol 27,11, pp. 1134–1142, 1984.
Y. Zhang. User’s guide to LIPSOL, Rice University, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag London
About this paper
Cite this paper
Kowalczyk, W. (2004). An Approximate Algorithm for Reverse Engineering of Multi-layer Perceptrons. In: Coenen, F., Preece, A., Macintosh, A. (eds) Research and Development in Intelligent Systems XX. SGAI 2003. Springer, London. https://doi.org/10.1007/978-0-85729-412-8_5
Download citation
DOI: https://doi.org/10.1007/978-0-85729-412-8_5
Publisher Name: Springer, London
Print ISBN: 978-1-85233-780-3
Online ISBN: 978-0-85729-412-8
eBook Packages: Springer Book Archive