torchact.nn.SinLU

class torchact.nn.SinLU(a: float = 1.0, b: float = 1.0)[source]

Implementation of Sinu-Sigmoidal Linear Unit https://doi.org/10.3390/math10030337

\(\text{SinLU}(x) = (x+a\text{sin}bx) \cdot \sigma(x)\)

Parameters

a (float) – Amplitude of the sine. Default: 1.0
b (float) – Frequency of the sine. Default: 1.0

Examples::

>>> import torch, torchact
>>> m = torchact.nn.SinLU()
>>> input = torch.randn(1, 3)
>>> output = m(input)
>>> print(output)
tensor([[0.1167, 2.2457, 0.0896]], grad_fn=<MulBackward0>)