torchrecurrent.PeepholeLSTMCell
- class torchrecurrent.PeepholeLSTMCell(input_size, hidden_size, bias=True, activation_fn=<built-in method tanh of type object>, gate_activation_fn=<built-in method sigmoid of type object>, kernel_init=<function xavier_uniform_>, recurrent_kernel_init=<function xavier_uniform_>, peephole_kernel_init=<function normal_>, bias_init=<function zeros_>, recurrent_bias_init=<function zeros_>, device=None, dtype=None)
A Peephole LSTM cell with learnable peephole connections.
This LSTM variant adds element-wise “peephole” terms from the cell state into the input, forget, and output gates.
\[\begin{split}\mathbf{z}(t) &= \tanh\Bigl( \mathbf{W}_{ih}^{z}\,\mathbf{x}(t) + \mathbf{b}_{ih}^{z} + \mathbf{W}_{hh}^{z}\,\mathbf{h}(t-1) + \mathbf{b}_{hh}^{z} \Bigr), \\[6pt] \mathbf{i}(t) &= \sigma\Bigl( \mathbf{W}_{ih}^{i}\,\mathbf{x}(t) + \mathbf{b}_{ih}^{i} + \mathbf{W}_{hh}^{i}\,\mathbf{h}(t-1) + \mathbf{b}_{hh}^{i} + \mathbf{p}^{i}\circ\mathbf{c}(t-1) \Bigr), \\[6pt] \mathbf{f}(t) &= \sigma\Bigl( \mathbf{W}_{ih}^{f}\,\mathbf{x}(t) + \mathbf{b}_{ih}^{f} + \mathbf{W}_{hh}^{f}\,\mathbf{h}(t-1) + \mathbf{b}_{hh}^{f} + \mathbf{p}^{f}\circ\mathbf{c}(t-1) \Bigr), \\[6pt] \mathbf{c}(t) &= \mathbf{f}(t)\,\circ\,\mathbf{c}(t-1) \;+\;\mathbf{i}(t)\,\circ\,\mathbf{z}(t), \\[6pt] \mathbf{o}(t) &= \sigma\Bigl( \mathbf{W}_{ih}^{o}\,\mathbf{x}(t) + \mathbf{b}_{ih}^{o} + \mathbf{W}_{hh}^{o}\,\mathbf{h}(t-1) + \mathbf{b}_{hh}^{o} + \mathbf{p}^{o}\circ\mathbf{c}(t) \Bigr), \\[6pt] \mathbf{h}(t) &= \mathbf{o}(t)\,\circ\,\tanh\bigl(\mathbf{c}(t)\bigr)\end{split}\]- Parameters:
input_size (int) – Number of input features \(\dim(\mathbf{x}(t))\).
hidden_size (int) – Number of hidden units \(\dim(\mathbf{h}(t))\).
bias (bool, optional) – If
False
, disables all biases \(\mathbf{b}_{ih}\) and \(\mathbf{b}_{hh}\). Default:True
.activation_fn (Callable, optional) – Activation for the cell candidate \(\mathbf{z}\). Default:
torch.tanh
.gate_activation_fn (Callable, optional) – Activation for input/forget/output gates. Default:
torch.sigmoid
.kernel_init (Callable, optional) – Initializer for input-to-hidden weights \(\mathbf{W}_{ih}\). Default:
nn.init.xavier_uniform_
.recurrent_kernel_init (Callable, optional) – Initializer for hidden-to-hidden weights \(\mathbf{W}_{hh}\). Default:
nn.init.xavier_uniform_
.peephole_kernel_init (Callable, optional) – Initializer for peephole weights \(\mathbf{p}\). Default:
nn.init.normal_
.bias_init (Callable, optional) – Initializer for input biases \(\mathbf{b}_{ih}\) when bias=True. Default:
nn.init.zeros_
.recurrent_bias_init (Callable, optional) – Initializer for hidden biases \(\mathbf{b}_{hh}\) when bias=True. Default:
nn.init.zeros_
.device (torch.device, optional) – Device of the parameters. Default: CPU.
dtype (torch.dtype, optional) – Data type of the parameters. Default: PyTorch float.
- Inputs: input, (h, c)
input (Tensor): shape (H_in,) or (N, H_in), where H_in = input_size.
- h, c (Tensor): previous hidden and cell states of shape
(H_out,) or (N, H_out), where H_out = hidden_size. If not provided, both default to zero.
- Outputs: h’, c’
h’, c’ (Tensor): next hidden and cell states, same shapes as inputs.
- Shape:
input: \((N, H_{\mathrm{in}})\) or \((H_{\mathrm{in}})\).
h, c: \((N, H_{\mathrm{out}})\) or \((H_{\mathrm{out}})\).
output: same as h and c.
- weight_ih
input‐to‐hidden weights, shape (4*H, I).
- Type:
Tensor
- weight_hh
hidden‐to‐hidden weights, shape (4*H, H).
- Type:
Tensor
- weight_ph
peephole weights, shape (3*H,) for i, f, o.
- Type:
Tensor
- bias_ih
input biases, shape (4*H,) if bias=True.
- Type:
Tensor
- bias_hh
hidden biases, shape (4*H,) if bias=True.
- Type:
Tensor
- Examples::
>>> cell = PeepholeLSTMCell(10, 20) >>> x = torch.randn(5, 10) >>> h, c = torch.zeros(20), torch.zeros(20) >>> for t in range(x.size(0)): ... h, c = cell(x[t], (h, c))
- __init__(input_size, hidden_size, bias=True, activation_fn=<built-in method tanh of type object>, gate_activation_fn=<built-in method sigmoid of type object>, kernel_init=<function xavier_uniform_>, recurrent_kernel_init=<function xavier_uniform_>, peephole_kernel_init=<function normal_>, bias_init=<function zeros_>, recurrent_bias_init=<function zeros_>, device=None, dtype=None)
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- Parameters:
input_size (int)
hidden_size (int)
bias (bool)
activation_fn (Callable)
gate_activation_fn (Callable)
device (device | None)
dtype (dtype | None)
Methods
__init__
(input_size, hidden_size[, bias, ...])Initialize internal Module state, shared by both nn.Module and ScriptModule.
add_module
(name, module)Add a child module to the current module.
apply
(fn)Apply
fn
recursively to every submodule (as returned by.children()
) as well as self.bfloat16
()Casts all floating point parameters and buffers to
bfloat16
datatype.buffers
([recurse])Return an iterator over module buffers.
children
()Return an iterator over immediate children modules.
compile
(*args, **kwargs)Compile this Module's forward using
torch.compile()
.cpu
()Move all model parameters and buffers to the CPU.
cuda
([device])Move all model parameters and buffers to the GPU.
double
()Casts all floating point parameters and buffers to
double
datatype.eval
()Set the module in evaluation mode.
extra_repr
()Return the extra representation of the module.
float
()Casts all floating point parameters and buffers to
float
datatype.forward
(inp[, state])Run one step of the recurrent cell.
get_buffer
(target)Return the buffer given by
target
if it exists, otherwise throw an error.get_extra_state
()Return any extra state to include in the module's state_dict.
get_parameter
(target)Return the parameter given by
target
if it exists, otherwise throw an error.get_submodule
(target)Return the submodule given by
target
if it exists, otherwise throw an error.half
()Casts all floating point parameters and buffers to
half
datatype.init_weights
()ipu
([device])Move all model parameters and buffers to the IPU.
load_state_dict
(state_dict[, strict, assign])Copy parameters and buffers from
state_dict
into this module and its descendants.modules
()Return an iterator over all modules in the network.
mtia
([device])Move all model parameters and buffers to the MTIA.
named_buffers
([prefix, recurse, ...])Return an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.
named_children
()Return an iterator over immediate children modules, yielding both the name of the module as well as the module itself.
named_modules
([memo, prefix, remove_duplicate])Return an iterator over all modules in the network, yielding both the name of the module as well as the module itself.
named_parameters
([prefix, recurse, ...])Return an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.
parameters
([recurse])Return an iterator over module parameters.
register_backward_hook
(hook)Register a backward hook on the module.
register_buffer
(name, tensor[, persistent])Add a buffer to the module.
register_forward_hook
(hook, *[, prepend, ...])Register a forward hook on the module.
register_forward_pre_hook
(hook, *[, ...])Register a forward pre-hook on the module.
register_full_backward_hook
(hook[, prepend])Register a backward hook on the module.
register_full_backward_pre_hook
(hook[, prepend])Register a backward pre-hook on the module.
register_load_state_dict_post_hook
(hook)Register a post-hook to be run after module's
load_state_dict()
is called.register_load_state_dict_pre_hook
(hook)Register a pre-hook to be run before module's
load_state_dict()
is called.register_module
(name, module)Alias for
add_module()
.register_parameter
(name, param)Add a parameter to the module.
register_state_dict_post_hook
(hook)Register a post-hook for the
state_dict()
method.register_state_dict_pre_hook
(hook)Register a pre-hook for the
state_dict()
method.requires_grad_
([requires_grad])Change if autograd should record operations on parameters in this module.
set_extra_state
(state)Set extra state contained in the loaded state_dict.
set_submodule
(target, module[, strict])Set the submodule given by
target
if it exists, otherwise throw an error.share_memory
()See
torch.Tensor.share_memory_()
.state_dict
(*args[, destination, prefix, ...])Return a dictionary containing references to the whole state of the module.
to
(*args, **kwargs)Move and/or cast the parameters and buffers.
to_empty
(*, device[, recurse])Move the parameters and buffers to the specified device without copying storage.
train
([mode])Set the module in training mode.
type
(dst_type)Casts all parameters and buffers to
dst_type
.uses_double_state
()Return True if forward returns (h, c), else just h.
xpu
([device])Move all model parameters and buffers to the XPU.
zero_grad
([set_to_none])Reset gradients of all model parameters.
Attributes
T_destination
call_super_init
dump_patches
training