This paper describes a novel memristor-based neuromorphic circuit that can be used for ex-situ training of multi-layer neural network algorithms. The presented ex-situ programming technique can be used to map many key neural algorithms directly onto the grid of resistances in a memristor crossbar. This is possible because the proposed circuit is capable of calculating a true dot product. Existing circuits provide an approximated dot product based on a summation of conductance values, which is inaccurate due to the parallel structure of the crossbar. To show the effectiveness and versatility of this circuit, three different powerful neural networks were simulated. These include a Restricted Boltzmann Machine (RBM) for character recognition, a Convolutional Neural Network (CNN) also for character recognition, and a Multilayer Perceptron (MLP) trained to perform Sobel edge detection. Finally, an architecture analysis was performed showing that neuromorphic processors based on these memristor crossbar circuits can be up to 5 orders of magnitude more power efficient than a RISC processor.