Bidirectional is a "wrapper" layer: it wraps any uni-directional RNN layer to make it bidirectional.
Note that multiple different modes are supported - these specify how the activations should be combined from the forward and backward RNN networks.
LastTimeStep is a "wrapper" layer: it wraps any RNN layer, and extracts out the last time step during forward pass, and returns it as a row vector (per example).
Simple RNN - aka "vanilla" RNN is the simplest type of recurrent neural network layer.
This Mode enumeration defines how the activations for the forward and backward networks should be combined.
ADD: out = forward + backward (elementwise addition)
MUL: out = forward * backward (elementwise multiplication)
AVERAGE: out = 0.5 * (forward + backward)
CONCAT: Concatenate the activations.
Where 'forward' is the activations for the forward RNN, and 'backward' is the activations for the backward RNN.