Spatially high-resolution information on the seabed sediment is import for many applications in the fields of oceanic engineering, coastal engineering, habitat mapping, and others. The seabed sediment is typically described by information based on the grain-size distribution, which are derived from sediment samples collected from the seafloor. For covering large areas side-scan sonar systems are typically used, which measure the backscatter intensity. From this information the sediment types can be derived. We propose a model for the automatic sediment type classification of the side-scan sonar data, which is based on convolutional neural networks (CNN). A big advantage of CNN is that they provide an end-to-end training: the CNN derives appropriate features automatically during the training process, which are then used for classification. The approach is based on a patch-wise classification using ensemble voting. The approach is evaluated on real world side-scan sonar data, which have been labelled using four classes (fine, sand, coarse, and mixed sediment) by experts. While the prediction of sand achieves an accuracy of 83 percent, the accuracy for fine sediment is very poor (11 percent).