RecurrentLayers

RecurrentLayers.jl extends Flux.jl recurrent layers offering by providing implementations of additional recurrent layers not available in base deep learning libraries.

Features

The package offers multiple layers for Flux.jl. Currently there are 30+ cells implemented, together with multiple higher level implementations:

Short namePublication venueOfficial implementation
AntisymmetricRNN/GatedAntisymmetricRNNICLR 2019
ATREMNLP 2018bzhangGo/ATR
BR/BRCPLOS ONE 2021nvecoven/BRC
CFNICLR 2017
coRNNICLR 2021tk-rusch/coRNN
FastRNN/FastGRNNNeurIPS 2018Microsoft/EdgeML
FSRNNNeurIPS 2017amujika/Fast-Slow-LSTM
IndRNNCVPR 2018Sunnydreamrain/IndRNNTheanoLasagne
JANETarXiv 2018JosvanderWesthuizen/janet
LEMICLR 2022tk-rusch/LEM
LiGRUIEEE Transactions on Emerging Topics in Computing 2018mravanelli/theano-kaldi-rnn
LightRUMDPI Electronics 2023
MinimalRNNNeurIPS 2017
MultiplicativeLSTMWorkshop ICLR 2017benkrause/mLSTM
MGUInternational Journal of Automation and Computing 2016
MUT1/MUT2/MUT3ICML 2015
NASarXiv 2016tensorflow_addons/rnn
PeepholeLSTMJMLR 2002
RANarXiv 2017kentonl/ran
RHNICML 2017jzilly/RecurrentHighwayNetworks
SCRNICLR 2015facebookarchive/SCRNNs
SGRNIET 2018
STARIEEE Transactions on Pattern Analysis and Machine Intelligence 20220zgur0/STAckable-Recurrent-network
Typed RNN / GRU / LSTMICML 2016
UnICORNNICML 2021tk-rusch/unicornn
WMCLSTMNeural Networks 2021

Installation

You can install RecurrentLayers using either of:

using Pkg
Pkg.add("RecurrentLayers")
julia> ]
pkg> add RecurrentLayers

Contributing

Contributions are always welcome! We specifically look for :

  • Recurrent cells you would like to see implemented
  • Benchmarks
  • Fixes for any bugs/errors
  • Documentation, in any form: examples, how tos, docstrings

Please consider the following guidelines before opening a pull request:

  • The code should be formatted according to the format file provided
  • Variable names should be meaningful: please no single letter variables, and try to avoid double letters variables too. I know at the moment there are some in the codebase, but I will need a breaking change in order to fix the majority of them.
  • The format file does not format markdown. If you are adding docs, or docstrings please take care of not going over 92 cols.

For any clarification feel free to contact me directly (@MartinuzziFrancesco) either in the julia slack, by email or X/bluesky.