FSRNNCell

FSRNNCell(input_size => hidden_size,
    fast_cells, slow_cell)

Fast slow recurrent neural network cell ^[Mujika2017]. See FSRNN for a layer that processes entire sequences.

Arguments

• input_size => hidden_size: input and inner dimension of the layer
• fast_cells: a vector of the fast cells. Must be minimum of length 2.
• slow_cell: the chosen slow cell.

Equations

\[\begin{aligned} \mathbf{h}^{F_1}(t) &= f^{F_1}\left( \mathbf{h}^{F_k}(t-1), \mathbf{x}(t) \right), \\ \mathbf{h}^{S}(t) &= f^{S}\left( \mathbf{h}^{S}(t-1), \mathbf{h}^{F_1}(t) \right), \\ \mathbf{h}^{F_2}(t) &= f^{F_2}\left( \mathbf{h}^{F_1}(t), \mathbf{h}^{S}(t) \right), \\ \mathbf{h}^{F_i}(t) &= f^{F_i}\left( \mathbf{h}^{F_{i-1}}(t) \right) \quad \text{for } 3 \leq i \leq k \end{aligned}\]

Forward

fsrnncell(inp, (fast_state, slow_state))
fsrnncell(inp)

Arguments

• inp: The input to the fsrnncell. It should be a vector of size input_size or a matrix of size input_size x batch_size.
• (fast_state, slow_state): A tuple containing the hidden and cell states of the FSRNNCell. They should be vectors of size hidden_size or matrices of size hidden_size x batch_size. If not provided, they are assumed to be vectors of zeros, initialized by Flux.initialstates.

Returns

• A tuple (output, state), where output = new_state is the new hidden state and state = (fast_state, slow_state) is the new hidden and cell state. They are tensors of size hidden_size or hidden_size x batch_size.