The approximation algorithm introduced by Asim Roy et al. (1997) generates a hybrid neural network with RBF neurons and other types of hidden neurons for function approximation. The network is trained in stages, with RBF neurons at the early stages corresponding to general features in the space and those in later stages corresponding to more specific features. The other types of hidden neurons are added with a view to improving generalization and reducing the number of RBF neurons. The algorithm uses linear programming to design and train the hybrid network. We investigate simplifying the algorithm with a view to eliminating the need for the other types of hidden neurons and linear programming. The simple hierarchical approximation algorithm ('SHA') achieves comparable results in terms of accuracy without the added complexity introduced by the other types of hidden neurons.