LayerNormalization
class LayerNormalization<T : DType, V>(normalizedShape: IntArray, eps: Double = 1.0E-5, elementwiseAffine: Boolean = true, val name: String = "LayerNormalization", initGamma: Tensor<T, V>? = null, initBeta: Tensor<T, V>? = null) : Module<T, V> , ModuleParameters<T, V> (source)
LayerNormalization layer - Used in attention mechanisms. Normalizes the input across the last dimension(s).
Parameters
normalizedShape
The shape of the normalization (typically the last dimension(s))
eps
Small value added to the denominator for numerical stability
elementwiseAffine
Whether to learn elementwise affine parameters (gamma and beta)
name
Name of the module
initGamma
Initial gamma (scale) parameter
initBeta
Initial beta (shift) parameter