A major challenge for conventional machine learning algorithms is performing intelligent fault diagnosis based on time domain raw vibrational data, where there are no explicitly defined high-level features. Studying and developing methods which work directly on the raw time data is beneficial to the scientific community since it reduces the time spent in the feature engineering process. In this paper, deep learning approaches are introduced for fault diagnosis in electrical submersible pumps (ESP) using raw time-domain data. Three neural network architectures are presented for classifying signals from this domain, an architecture based on a conventional algorithm (Multi Layer Perceptron), a traditional convolutional architecture and a convolutional architecture using a triplet loss function with the goal of generating a space of latent features to perform a classification using a conventional K-nearest neighbor(KNN) algorithm. For comparison purposes, two baseline methods are used. The first baseline is a state-of-the-art method for ESP fault diagnosis that uses data from the frequency domain. It is used as a higher bound baseline. The second baseline uses feature engineering and traditional time signal techniques, serving as a lower performance bound to the new proposed methods. The results show that one of the proposed methods is able to diagnose effectively with macro F-measure of 0.65, a much better result than the lower bound method and not too far from the upper bound method.