Pytorch Notes

Pytorch Notes

1. Be aware of the tensor shape. The shape of torch.tensor([1]) and torch.tensor(1) is different. One is torch.Size([1]), the other is torch.Size([]).

Note that the output shape of nn.Linear(input_dim, 1) is not torch.Size([]); It is torch.Size([1]). If we want that output to do some caculation with scalar tensor of shape torch.Size([]), we should do x = x.squeeze(0).

2. When caculating loss, always place pred as the first argument and label as the second arg.

3. When calulating CrossEntropy, we can use the class number(scalar) as label and tensor with shape(batch_size, num_of_all_classes) as prediction.

4. Remember to flatten the tensor to shape(batch_size, num) before applying nn.Linear(num, num2).

5. torch.argmax(t) will decrement the dimension number of t by one.

6. If dataset returns a tuple (tensor with shape(x1, y1), tensor with shape(x2, y2)), corresponding dataloader returns [tensor with shape(batch_size, x1, y1), tensor with shape(batch_size, x2, y2)].