Pytorch Linear ()简单推导

pytorch,nn.Linear

下图中的A是权重矩阵，b是偏置。

• in_features输入样本的张量大小
• out_features输出样本的张量大小
• bias是偏置

测试代码

• batch_size : 输入样例数
• in_features : 输入样例特征数
• out_features : 输出样例特征数

fc     = nn.Linear(3, 3)     # [in_features, out_features]
tensor = torch.randn(4, 3)   # [batch_size, in_features]
output = fc(tensor)          # [batch_size, out_features]

print(fc.weight.shape)
print(fc.bias.shape)
print(output.size())

输出

torch.Size([3, 3])
torch.Size([3])
torch.Size([4, 3])

这里我们假设一下AT,x,b

import numpy as np
AT = np.mat([[1, 2, 3],
            [2, 1, 3],
            [3, 1, 2],
            [3, 2, 1]])

x = np.mat([[0.1, 0.2, 0.3],
            [0.2, 0.1, 0.3],
            [0.3, 0.2, 0.1]])

b = np.mat([1, 1, 1])

输入数据x经过线性变换得到结果如下:

print((AT * x) + b)

输出

[[2.4 2.  2.2]
 [2.3 2.1 2.2]
 [2.1 2.1 2.4]
 [2.  2.  2.6]]

输入输出的张量的shape角度来理解，相当于一个输入为[batch_size, in_features]的张量变换成了[batch_size, out_features]的输出张量。