This paper proposes a task decomposition method, which divides a large-scale learning problem into multiple limited-scale pairs of training subsets and cross validation (CV) subsets. Correspondingly, modular multilayer perceptrons are set up. At first, one training subset only consists of its own class and several most neighboring categories, and then some classes in the CV subset are moved into it according to the generalization error of the module. This work presents an empirical formula for selecting the initial number of hidden nodes, and a method for determining the optimal number of hidden units with the help of singular value decomposition. The result for letter recognition shows that the above methods are quite effective.