Deep learning model with ensemble techniques to compute the secondary structure of proteins

Abstract

Protein secondary structure is the local conformation assigned to protein sequences with the help of its three-dimensional structure. Assigning the local conformation to protein sequences requires much computational work. There exists a vast literature on the protein secondary structure prediction approaches (more than 20 techniques), but to date, none of the existing techniques is entirely accurate. Thus, there is an excellent room for developing new models of protein secondary structure prediction to address the issues of prediction accuracy. In the present study, ensemble techniques such as AdaBoost- and Bagging-based deep learning models are proposed to predict the protein secondary structure. The data from standard datasets, namely CB513, RS126, PTOP742, PSA472, and MANESH, have been used for training and testing purposes. These standard datasets possess less than 25% redundancy. The model is evaluated using performance measures: Q8 and Q3 cross-validation accuracy, class precision, class recall, kappa factor, and testing on a dataset that is not used for training purposes, i.e., blind test. The ensembling technique used along with variability in datasets can remove the bias of each dataset by balancing it and making the features more distinguishable, leading to the improvement in accuracy as compared to the conventional and existing techniques. The proposed model shows an average improvement of ~ 2% and ~ 3% accuracy over the existing methods in a blind test for Q8 and Q3 accuracy.

Appendix

1.1 Experimental set up

A computational framework has been proposed for prediction of secondary structure of proteins. The approaches employed to construct the computational framework are: supervised learning, simple deep learning, and deep learning with ensemble techniques. The steps of the proposed computational framework are given below:

• 1. Preparation of dataset.

• 2. Cleansing of dataset.

• 3. Construction of deep neural network.

  • Feed-forward neural network

  • Backpropagation

• 4.Deep neural network with ensemble techniques

  • Bagging

  • AdaBoost

• 5. Training.

• 6. Testing.

  • Cross-validation

  • Blind test

Multilayer feed-forward-based artificial deep neural network is applied and trained with stochastic gradient descent (SGD). The stochastic gradient descent (SGD) algorithm is a drastic simplification of simple gradient descent. Instead of computing the gradient of En(fw) exactly, each iteration estimates this gradient on the basis of a single randomly picked example [53]. A deep neural network containing two layers each having 100 nodes is trained with rectifier activation function.

Advanced features such as adaptive learning rate, rate annealing, momentum training, dropout, and L1 or L2 regularization enable high predictive accuracy. Each compute node trains a copy of the global model parameters on its local data with multi-threading (asynchronously) and contributes periodically to the global model via model averaging across the network.

A multilayer deep neural network can distort the input space to make the classes of data (examples of which are on the red and blue lines). Note how a regular grid (shown on the left) in input space is also transformed (shown in the middle panel) by hidden units.

The chain rule of derivatives tells how two small effects (that of a small change of x on y, and that of y on z) are composed. A small change Δx in x gets transformed first into a small change Δy in y by getting multiplied by ∂y/∂x. Similarly, the change Δy creates a change Δz in z. On substituting one equation into the other gives the chain rule of derivatives—how Δx gets turned into Δz through multiplication by the product of ∂y/∂x and ∂z/∂x.

$$\Delta z = \frac{\partial z}{{\partial y}}\Delta y$$

(5)

$$\Delta y = \frac{\partial y}{{\partial x}}\Delta x$$

(6)

$$\Delta z = \frac{\partial z}{{\partial y}}\frac{\partial y}{{\partial x}}\Delta x$$

(7)

$$\frac{\partial z}{{\partial x}} = \frac{\partial z}{{\partial y}}\frac{\partial y}{{\partial x}}$$

(8)

See Fig. 

Fig. 2

Feed-forward deep neural network

2

The equations used for computing the forward pass in a deep neural net with two hidden layers and one output layer, each constituting a module through which one can backpropagate gradients, are shown in Fig. 2. At each layer, the total input z is computed first to each unit, which is a weighted sum of the outputs of the units in the layer below to the previous layer. Then a nonlinear function f(.) is applied to z to get the output of the unit. For simplicity, in Fig. 2 bias terms are omitted. The nonlinear functions used in neural networks include the rectified linear unit (ReLU) f(z) = max(0,z). This nonlinear function is commonly used in recent years [7] (see Fig. 

Fig. 3

Backpropagation

3).

The backward pass is computed by the equations are shown in Fig. 3. At each hidden layer, the error derivative is computed with respect to the output of each unit, which is a weighted sum of the error derivatives with respect to the total inputs to the units in the layer above. The error derivative is then converted with respect to the output into the error derivative with respect to the input by multiplying it by the gradient of f(z). At the output layer, the error derivative with respect to the output of a unit is computed by differentiating the cost function [7].