torchrecurrent.PeepholeLSTMCell#

class torchrecurrent.PeepholeLSTMCell(input_size, hidden_size, bias=True, recurrent_bias=True, nonlinearity=<built-in method tanh of type object>, gate_nonlinearity=<built-in method sigmoid of type object>, kernel_init=<function xavier_uniform_>, recurrent_kernel_init=<function xavier_uniform_>, peephole_kernel_init=<function normal_>, bias_init=<function zeros_>, recurrent_bias_init=<function zeros_>, device=None, dtype=None)[source]#

A Peephole LSTM cell with learnable peephole connections.

[JMLR]

\[\begin{split}\mathbf{z}(t) &= \tanh\Bigl( \mathbf{W}_{ih}^{z}\,\mathbf{x}(t) + \mathbf{b}_{ih}^{z} + \mathbf{W}_{hh}^{z}\,\mathbf{h}(t-1) + \mathbf{b}_{hh}^{z} \Bigr), \\[6pt] \mathbf{i}(t) &= \sigma\Bigl( \mathbf{W}_{ih}^{i}\,\mathbf{x}(t) + \mathbf{b}_{ih}^{i} + \mathbf{W}_{hh}^{i}\,\mathbf{h}(t-1) + \mathbf{b}_{hh}^{i} + \mathbf{p}^{i}\circ\mathbf{c}(t-1) \Bigr), \\[6pt] \mathbf{f}(t) &= \sigma\Bigl( \mathbf{W}_{ih}^{f}\,\mathbf{x}(t) + \mathbf{b}_{ih}^{f} + \mathbf{W}_{hh}^{f}\,\mathbf{h}(t-1) + \mathbf{b}_{hh}^{f} + \mathbf{p}^{f}\circ\mathbf{c}(t-1) \Bigr), \\[6pt] \mathbf{c}(t) &= \mathbf{f}(t)\,\circ\,\mathbf{c}(t-1) + \mathbf{i}(t)\,\circ\,\mathbf{z}(t), \\[6pt] \mathbf{o}(t) &= \sigma\Bigl( \mathbf{W}_{ih}^{o}\,\mathbf{x}(t) + \mathbf{b}_{ih}^{o} + \mathbf{W}_{hh}^{o}\,\mathbf{h}(t-1) + \mathbf{b}_{hh}^{o} + \mathbf{p}^{o}\circ\mathbf{c}(t) \Bigr), \\[6pt] \mathbf{h}(t) &= \mathbf{o}(t)\,\circ\,\tanh\bigl(\mathbf{c}(t)\bigr)\end{split}\]

Parameters:

input_size – The number of expected features in the input x.
hidden_size – The number of features in the hidden state h.
bias – If False, the layer does not use input-side bias b_{ih}. Default: True.
recurrent_bias – If False, the layer does not use recurrent bias b_{hh}. Default: True.
nonlinearity – Activation for the cell candidate z. Default: torch.tanh().
gate_nonlinearity – Activation for input/forget/output gates. Default: torch.sigmoid().
kernel_init – Initializer for W_{ih}. Default: torch.nn.init.xavier_uniform_().
recurrent_kernel_init – Initializer for W_{hh}. Default: torch.nn.init.xavier_uniform_().
peephole_kernel_init – Initializer for peephole weights p. Default: torch.nn.init.normal_().
bias_init – Initializer for b_{ih} when bias=True. Default: torch.nn.init.zeros_().
recurrent_bias_init – Initializer for b_{hh} when recurrent_bias=True. Default: torch.nn.init.zeros_().
device – The desired device of parameters.
dtype – The desired floating point type of parameters.

Inputs: input, (h_0, c_0)

input of shape (batch, input_size) or (input_size,): Tensor containing input features.
h_0 of shape (batch, hidden_size) or (hidden_size,): Tensor containing the initial hidden state.
c_0 of shape (batch, hidden_size) or (hidden_size,): Tensor containing the initial cell state.

If (h_0, c_0) is not provided, both default to zero.

Outputs: (h_1, c_1)

h_1 of shape (batch, hidden_size) or (hidden_size,): Tensor containing the next hidden state.
c_1 of shape (batch, hidden_size) or (hidden_size,): Tensor containing the next cell state.

Variables:

weight_ih – The learnable input–hidden weights, of shape (4*hidden_size, input_size).
weight_hh – The learnable hidden–hidden weights, of shape (4*hidden_size, hidden_size).
weight_ph – The learnable peephole weights (for i, f, o), of shape (3*hidden_size,).
bias_ih – The learnable input–hidden biases, of shape (4*hidden_size).
bias_hh – The learnable hidden–hidden biases, of shape (4*hidden_size).

Examples:

>>> cell = PeepholeLSTMCell(10, 20)
>>> x = torch.randn(5, 3, 10)   # (time_steps, batch, input_size)
>>> h = torch.zeros(3, 20)
>>> c = torch.zeros(3, 20)
>>> outs = []
>>> for t in range(x.size(0)):
...     h, c = cell(x[t], (h, c))
...     outs.append(h)
>>> outs = torch.stack(outs, dim=0)  # (time_steps, batch, hidden_size)

__init__(input_size, hidden_size, bias=True, recurrent_bias=True, nonlinearity=<built-in method tanh of type object>, gate_nonlinearity=<built-in method sigmoid of type object>, kernel_init=<function xavier_uniform_>, recurrent_kernel_init=<function xavier_uniform_>, peephole_kernel_init=<function normal_>, bias_init=<function zeros_>, recurrent_bias_init=<function zeros_>, device=None, dtype=None)[source]#: Initialize internal Module state, shared by both nn.Module and ScriptModule.

Methods

`__init__`(input_size, hidden_size[, bias, ...])	Initialize internal Module state, shared by both nn.Module and ScriptModule.
`add_module`(name, module)	Add a child module to the current module.
`apply`(fn)	Apply `fn` recursively to every submodule (as returned by `.children()`) as well as self.
`bfloat16`()	Casts all floating point parameters and buffers to `bfloat16` datatype.
`buffers`([recurse])	Return an iterator over module buffers.
`children`()	Return an iterator over immediate children modules.
`compile`(args, *kwargs)	Compile this Module's forward using `torch.compile()`.
`cpu`()	Move all model parameters and buffers to the CPU.
`cuda`([device])	Move all model parameters and buffers to the GPU.
`double`()	Casts all floating point parameters and buffers to `double` datatype.
`eval`()	Set the module in evaluation mode.
`extra_repr`()	Return the extra representation of the module.
`float`()	Casts all floating point parameters and buffers to `float` datatype.
`forward`(inp[, state])	Run one step of the recurrent cell.
`get_buffer`(target)	Return the buffer given by `target` if it exists, otherwise throw an error.
`get_extra_state`()	Return any extra state to include in the module's state_dict.
`get_parameter`(target)	Return the parameter given by `target` if it exists, otherwise throw an error.
`get_submodule`(target)	Return the submodule given by `target` if it exists, otherwise throw an error.
`half`()	Casts all floating point parameters and buffers to `half` datatype.
`init_weights`()
`ipu`([device])	Move all model parameters and buffers to the IPU.
`load_state_dict`(state_dict[, strict, assign])	Copy parameters and buffers from `state_dict` into this module and its descendants.
`modules`()	Return an iterator over all modules in the network.
`mtia`([device])	Move all model parameters and buffers to the MTIA.
`named_buffers`([prefix, recurse, ...])	Return an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.
`named_children`()	Return an iterator over immediate children modules, yielding both the name of the module as well as the module itself.
`named_modules`([memo, prefix, remove_duplicate])	Return an iterator over all modules in the network, yielding both the name of the module as well as the module itself.
`named_parameters`([prefix, recurse, ...])	Return an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.
`parameters`([recurse])	Return an iterator over module parameters.
`register_backward_hook`(hook)	Register a backward hook on the module.
`register_buffer`(name, tensor[, persistent])	Add a buffer to the module.
`register_forward_hook`(hook, *[, prepend, ...])	Register a forward hook on the module.
`register_forward_pre_hook`(hook, *[, ...])	Register a forward pre-hook on the module.
`register_full_backward_hook`(hook[, prepend])	Register a backward hook on the module.
`register_full_backward_pre_hook`(hook[, prepend])	Register a backward pre-hook on the module.
`register_load_state_dict_post_hook`(hook)	Register a post-hook to be run after module's `load_state_dict()` is called.
`register_load_state_dict_pre_hook`(hook)	Register a pre-hook to be run before module's `load_state_dict()` is called.
`register_module`(name, module)	Alias for `add_module()`.
`register_parameter`(name, param)	Add a parameter to the module.
`register_state_dict_post_hook`(hook)	Register a post-hook for the `state_dict()` method.
`register_state_dict_pre_hook`(hook)	Register a pre-hook for the `state_dict()` method.
`requires_grad_`([requires_grad])	Change if autograd should record operations on parameters in this module.
`set_extra_state`(state)	Set extra state contained in the loaded state_dict.
`set_submodule`(target, module[, strict])	Set the submodule given by `target` if it exists, otherwise throw an error.
`share_memory`()	See `torch.Tensor.share_memory_()`.
`state_dict`(*args[, destination, prefix, ...])	Return a dictionary containing references to the whole state of the module.
`to`(args, *kwargs)	Move and/or cast the parameters and buffers.
`to_empty`(*, device[, recurse])	Move the parameters and buffers to the specified device without copying storage.
`train`([mode])	Set the module in training mode.
`type`(dst_type)	Casts all parameters and buffers to `dst_type`.
`uses_double_state`()	Return True if forward returns (h, c), else just h.
`xpu`([device])	Move all model parameters and buffers to the XPU.
`zero_grad`([set_to_none])	Reset gradients of all model parameters.

Attributes

`T_destination`
`call_super_init`
`dump_patches`
`weight_ih`
`weight_hh`
`weight_ph`
`bias_ih`
`bias_hh`
`input_size`
`hidden_size`
`bias`
`recurrent_bias`
`training`