RecurrentLayers

RecurrentLayers.jl extends Flux.jl recurrent layers offering by providing implementations of additional recurrent layers not available in base deep learning libraries.

Features

The package offers multiple layers for Flux.jl. Currently there are 30+ cells implemented, together with multiple higher level implementations:

• Modifications of vanilla RNNs: Independently recurrent neural networks, Structurally constrained recurrent neural network, FastRNN, and Typed RNNs.

• Variations over gated architectures: Minimal gated unit, Light gated recurrent networks, Recurrent addictive networks, Light recurrent networks, Neural architecture search networks, Evolving recurrent neural networks, Peephole long short term memory, FastGRNN, Just another network, Chaos free network, Typed gated recurrent unit, Typed long short term memory, Stackable recurrent network, Minimal recurrent neural network, Addition-subtraction twin-gated recurrent cell, Simple gated recurrent network, Bistable recurrent cell, Recurrently neuromodulated bistable recurrent cell, Peephole long short term memory cell, and Long short term memory cell with working memory connections.

• Discretized ordinary differential equation formulations of RNNs: Long expressive memory networks, Coupled oscillatory recurrent neural unit, Antisymmetric recurrent neural network with its gated version, and Undamped independent controlled oscillatory recurrent neural network.

• Additional more complex architectures: Recurrent highway networks, and FastSlow RNNs

• Additional wrappers: Stacked RNNs, and

Multiplicative RNN.

Installation

You can install RecurrentLayers using either of:

using Pkg
Pkg.add("RecurrentLayers")

julia> ]
pkg> add RecurrentLayers

Contributing

Contributions are always welcome! We specifically look for :

• Recurrent cells you would like to see implemented
• Benchmarks
• Fixes for any bugs/errors
• Documentation, in any form: examples, how tos, docstrings

Please consider the following guidelines before opening a pull request:

• The code should be formatted according to the format file provided
• Variable names should be meaningful: please no single letter variables, and try to avoid double letters variables too. I know at the moment there are some in the codebase, but I will need a breaking change in order to fix the majority of them.
• The format file does not format markdown. If you are adding docs, or docstrings please take care of not going over 92 cols.

For any clarification feel free to contact me directly (@MartinuzziFrancesco) either in the julia slack, by email or X/bluesky.