FSRNN

FSRNN(input_size => hidden_size,
    fast_cells, slow_cell;
    return_state=false)

Fast slow recurrent neural network ^[Mujika2017]. See FSRNNCell for a layer that processes a single sequence.

Arguments

• input_size => hidden_size: input and inner dimension of the layer.
• fast_cells: a vector of the fast cells. Must be minimum of length 2.
• slow_cell: the chosen slow cell.
• return_state: option to return the last state. Default is false.

Equations

\[\begin{aligned} \mathbf{h}^{F_1}(t) &= f^{F_1}\left( \mathbf{h}^{F_k}(t-1), \mathbf{x}(t) \right), \\ \mathbf{h}^{S}(t) &= f^{S}\left( \mathbf{h}^{S}(t-1), \mathbf{h}^{F_1}(t) \right), \\ \mathbf{h}^{F_2}(t) &= f^{F_2}\left( \mathbf{h}^{F_1}(t), \mathbf{h}^{S}(t) \right), \\ \mathbf{h}^{F_i}(t) &= f^{F_i}\left( \mathbf{h}^{F_{i-1}}(t) \right) \quad \text{for } 3 \leq i \leq k \end{aligned}\]

Forward

fsrnn(inp, (fast_state, slow_state))
fsrnn(inp)

Arguments

• inp: The input to the fsrnn. It should be a vector of size input_size or a matrix of size input_size x batch_size.
• (fast_state, slow_state): A tuple containing the hidden and cell states of the FSRNN. They should be vectors of size hidden_size or matrices of size hidden_size x batch_size. If not provided, they are assumed to be vectors of zeros, initialized by Flux.initialstates.

Returns

• New hidden states new_states as an array of size hidden_size x len x batch_size. When return_state = true it returns a tuple of the hidden stats new_states and the last state of the iteration.