When inferring nonlinear dependence from measured data, the nonlinear nature of the relationship may be characterised in terms of all the explanatory variables. However, this is rarely the most parsimonious, or insightful, approach. Instead, it is usually much more useful to be able to exploit the inherent nonlinear structure to characterise the nonlinear dependence in terms of the least possible number of variables. In this paper a new way of inferring nonlinear structure from measured data is investigated. The measured data is interpreted as providing information on a nonlinear map. The space containing the domain of the map is sub-divided into unique linear and nonlinear sub-spaces that are structural invariants. The most parsimonious representation of the map is obtained by the restriction of the map to the nonlinear sub-space. A direct constructive algorithm based on Gaussian process prior models, defined using a novel covariance function, is proposed. The algorithm infers the linear and nonlinear sub-spaces from noisy data and provides a non-parametric model of the parsimonious map. Use of the algorithm is illustrated by application to a Wiener-Hammerstein system.