SELU

class torch.nn.modules.activation.SELU(inplace=False)

Applies the SELU function element-wise.

$$\text{SELU}(x)=\text{scale}\ast (\max (0,x)+\min (0,\alpha \ast (\exp (x)-1)))$$

with $\alpha =1.6732632423543772848170429916717$ and $\text{scale}=1.0507009873554804934193349852946$.

Warning

When using kaiming_normal or kaiming_normal_ for initialisation, nonlinearity='linear' should be used instead of nonlinearity='selu' in order to get Self-Normalizing Neural Networks. See torch.nn.init.calculate_gain() for more information.

More details can be found in the paper Self-Normalizing Neural Networks .

Parameters

inplace (bool, optional) – can optionally do the operation in-place. Default: False

Shape:

• Input: $(\ast )$, where $\ast$ means any number of dimensions.

• Output: $(\ast )$, same shape as the input.

Examples:

>>> m = nn.SELU()
>>> input = torch.randn(2)
>>> output = m(input)

extra_repr()

Return the extra representation of the module.

Return type

str

forward(input)

Runs the forward pass.

Return type

Tensor