nn.MarginRankingLoss介绍

nn.MarginRankingLoss

复现论文代码中，它使用了MarginRankingLoss()函数，以下是我百度的内容：

排序损失函数

对于包含\(\mathbf{N}\)个样本的batch数据 \(D(x_1,x_2,y)\), \(x_1\),\(x_2\)是给定的待排序的两个输入，\(y\)代表真实的标签，属于{ 1 , − 1 } 。当Y = 1 是，\(x_1\)应该排在\(x_2\)前，Y = − 1 是，\(x_1\)应该排在\(x_2\)之后。

第n个样本对应的loss计算如下:

\[l_n = \max(0,-y*(x_1-x_2)+margin) \]

若\(x_1\)，\(x_2\)排序正确且\(-y*(x_1-x_2)>margin\)，则loss为0

class MarginRankingLoss(_Loss):
    __constants__ = ['margin', 'reduction']
    def __init__(self, margin=0., size_average=None, reduce=None, reduction='mean'):
        super(MarginRankingLoss, self).__init__(size_average, reduce, reduction)
        self.margin = margin
    def forward(self, input1, input2, target):
        return F.margin_ranking_loss(input1, input2, target, margin=self.margin, reduction=self.reduction)

pytorch中通过torch.nn.MarginRankingLoss类实现，也可以直接调用F.margin_ranking_loss 函数，代码中的size_average与reduce已经弃用。reduction有三种取值mean, sum, none，对应不同的返回ℓ ( x , y )。 默认为mean，对应于上述loss的计算

\[L=\{l_1,\dots, l_N\} \]

\[\ell(x, y)= \begin{cases}\mathrm{L}, & \text { if reduction = 'none' } \\ \frac{1}{N} \sum_{i=1}^{N} l_{i}, & \text { if reduction = 'mean' } \\ \sum_{i=1}^{N} l_{i} & \text { if reduction = 'sum' }\end{cases} \]

margin默认取0

例子：

import torch
import torch.nn.functional as F
import torch.nn as nn
import math

def validate_MarginRankingLoss(input1, input2, target, margin):
    val = 0
    for x1, x2, y in zip(input1, input2, target):
        loss_val = max(0, -y * (x1 - x2) + margin)
        val += loss_val
    return val / input1.nelement()

torch.manual_seed(10)
margin = 0
loss = nn.MarginRankingLoss()
input1 = torch.randn([3], requires_grad=True)
input2 = torch.randn([3], requires_grad=True)
target = torch.tensor([1, -1, -1])
print(target)
output = loss(input1, input2, target)
print(output.item())

output = validate_MarginRankingLoss(input1, input2, target, margin)
print(output.item())

loss = nn.MarginRankingLoss(reduction="none")
output = loss(input1, input2, target)
print(output)

'''
tensor([ 1, -1, -1])
0.015400052070617676
0.015400052070617676
tensor([0.0000, 0.0000, 0.0462], grad_fn=<ClampMinBackward>)
'''