Layer Types

Every neural network consists of multiple layers. Each layer has a variation of the following basic operation:

y = Φ(Wx + b)

where,

Φ = activation function
​W = weight matrix of the layer
​b = bias
​x = input
​y = output

Different activation layers perform a range of mathematical functions and are named as follows:

  1. Dense layer
  2. Convolution layer
  3. Pooling layer
  4. Recurrent layer (LSTM - Long Short Term Memory networks)