Linear

class torch.nn.Linear(in_features, out_features, bias=True, device=None, dtype=None)

Applies an affine linear transformation to the incoming data: $y=x{A}^T+b$.

This module supports TensorFloat32.

On certain ROCm devices, when using float16 inputs this module will use different precision for backward.

Parameters

• in_features (int) – size of each input sample

• out_features (int) – size of each output sample

• bias (bool) – If set to False, the layer will not learn an additive bias. Default: True

Shape:

• Input: $(\ast ,H_{\text{in}})$ where $\ast$ means any number of dimensions including none and $H_{\text{in}}=\text{in\_features}$.

• Output: $(\ast ,H_{\text{out}})$ where all but the last dimension are the same shape as the input and $H_{\text{out}}=\text{out\_features}$.

Variables

• weight (torch.Tensor) – the learnable weights of the module of shape $(\text{out\_features},\text{in\_features})$. The values are initialized from $\mathcal{U}(-\sqrt{k},\sqrt{k})$, where $k=\frac{1}{\text{in\_features}}$

• bias – the learnable bias of the module of shape $(\text{out\_features})$. If bias is True, the values are initialized from $\mathcal{U}(-\sqrt{k},\sqrt{k})$ where $k=\frac{1}{\text{in\_features}}$

Examples:

>>> m = nn.Linear(20, 30)
>>> input = torch.randn(128, 20)
>>> output = m(input)
>>> print(output.size())
torch.Size([128, 30])

extra_repr()

Return the extra representation of the module.

Return type

str

forward(input)

Runs the forward pass.

Return type

Tensor

reset_parameters()

Resets parameters based on their initialization used in __init__.