skainet-lang-core/sk.ainet.lang.nn.normalization/BatchNormalization

BatchNormalization

class BatchNormalization<T : DType, V>(numFeatures: Int, eps: Double = 1.0E-5, momentum: Double = 0.1, affine: Boolean = true, val name: String = "BatchNormalization", initGamma: Tensor<T, V>? = null, initBeta: Tensor<T, V>? = null) : Module<T, V> , ModuleParameters<T, V> (source)

BatchNormalization layer for training stability and performance. Normalizes the input across the batch dimension. https://arxiv.org/abs/1607.06450

Parameters

numFeatures

Number of features (channels)

eps

Small value added to the denominator for numerical stability

momentum

Momentum for running statistics update during training

affine

Whether to learn affine parameters (gamma and beta)

name

Name of the module

initGamma

Initial gamma (scale) parameter

initBeta

Initial beta (shift) parameter

Constructors

BatchNormalization

constructor(numFeatures: Int, eps: Double = 1.0E-5, momentum: Double = 0.1, affine: Boolean = true, name: String = "BatchNormalization", initGamma: Tensor<T, V>? = null, initBeta: Tensor<T, V>? = null)

Properties

modules

open override val modules: List<Module<T, V>>

name

open override val name: String

params

open override val params: List<ModuleParameter<T, V>>

Parameters owned by this node (weights, biases, etc.).

Functions

eval

fun eval()

Set the module to evaluation mode

forward

open override fun forward(input: Tensor<T, V>, ctx: ExecutionContext): Tensor<T, V>

train

fun train()

Set the module to training mode