关于 BCEloss的结果是否需要求平均

#1

我今天看了megengine的BCEloss的代码，发现源码在实现的时候求了均值，但是github上的代码却没有

image
image1434×847 72.1 KB

image
image1203×723 55.3 KB

下面是github上面的：
image
image2062×1142 146 KB

有点疑惑到底需不需要，pytorch的貌似没有 :joy:，说实话pytorch有没有求平均我也不太确定，找了它的半天源码在官网上，没找到 :joy:

#2

先提供答案：会自动求平均。

类似问题如何解决：搜到了 API 之后点击查看源码

def binary_cross_entropy(
    pred: Tensor, label: Tensor, with_logits: bool = True
) -> Tensor:
    r"""
    Computes the binary cross entropy loss (using logits by default).

    By default(``with_logitis`` is True), ``pred`` is assumed to be logits,
    class probabilities are given by sigmoid.

    :param pred: `(N, *)`, where `*` means any number of additional dimensions.
    :param label: `(N, *)`, same shape as the input.
    :param with_logits: bool, whether to apply sigmoid first. Default: True
    :return: loss value.

    Examples:

    .. testcode::

        import numpy as np
        from megengine import tensor
        import megengine.functional as F

        pred = tensor(np.array([0, 0], dtype=np.float32).reshape(1, 2))
        label = tensor(np.ones((1, 2), dtype=np.float32))
        loss = F.nn.binary_cross_entropy(pred, label)
        print(loss.numpy().round(decimals=4))

    Outputs:

    .. testoutput::

        0.6931

    """
    if not with_logits:
        return -(label * log(pred) + (1 - label) * log(1 - pred)).mean()
    # logsigmoid(pred) and logsigmoid(-pred) has common sub-expression
    # hopefully the backend would optimize this
    return -(label * logsigmoid(pred) + (1 - label) * logsigmoid(-pred)).mean()

可以看到有一个求 mean 的操作

#3

目前还没有提供相关的参数来支持自定义是否求 mean

不过你可以参考 Models 里面的实现：

https://github.com/MegEngine/Models/blob/master/official/vision/detection/layers/det/loss.py#L13

def binary_cross_entropy(
    pred: Tensor, label: Tensor, with_logits: bool = True
) -> Tensor:
    r"""
    Computes the binary cross entropy loss (using logits by default).

    By default(``with_logitis`` is True), ``pred`` is assumed to be logits,
    class probabilities are given by sigmoid.

    :param pred: `(N, *)`, where `*` means any number of additional dimensions.
    :param label: `(N, *)`, same shape as the input.
    :param with_logits: bool, whether to apply sigmoid first. Default: True
    :return: loss value.

    Examples:

    .. testcode::

        import numpy as np
        from megengine import tensor
        import megengine.functional as F

        pred = tensor(np.array([0, 0], dtype=np.float32).reshape(1, 2))
        label = tensor(np.ones((1, 2), dtype=np.float32))
        loss = F.nn.binary_cross_entropy(pred, label)
        print(loss.numpy().round(decimals=4))

    Outputs:

    .. testoutput::

        0.6931

    """
    if not with_logits:
        return -(label * log(pred) + (1 - label) * log(1 - pred)).mean()
    # logsigmoid(pred) and logsigmoid(-pred) has common sub-expression
    # hopefully the backend would optimize this
    return -(label * logsigmoid(pred) + (1 - label) * logsigmoid(-pred)).mean()
#4

行呢，我明白了:grin: