Reasoning is a high-level cognitive function that is gaining attention in the artificial neural network community. While there are many types of reasoning, this paper is specifically looking at valid categorical syllogisms. First we show that a standard bi-directional associative memory cannot learn all valid categorical syllogisms because these syllogisms are not linearly separable. Therefore a more complex architecture is proposed to learn the task. A combination of unsupervised and supervised learning networks are used. The unsupervised network compresses the input into novel solutions. The output from the unsupervised network in conjunction with the original input produces a new linearly separable input for the supervised network. This unsupervised-supervised learning network combination can successfully learn all the valid syllogisms. If there is a combination of valid and conditionally valid syllogisms, two different networks should be used. The conditionally valid syllogisms can be recalled using the bi-directional associative memory while the valid syllogisms need the more complex network.