dopt.nnet.parameters

This module contains methods for initialising the parameters of neural networks.

Several of the methods implemented in this module rely on fan_in and fan_out values. These are calculated differently depending on the rank of the parameter.

For rank-2 tensors,

fan_in = shape[0], fan_out = shape[1]

For rank-4 tensors,

fan_in = shape[1] * shape[2] * shape[3], fan_out = shape[0] * shape[2] * shape[3]

Members

Aliases

ParamInitializer
alias ParamInitializer = void delegate(Operation param)

Used to initialize a parameter in the neural network.

Functions

constantInit
ParamInitializer constantInit(float val)

Creates a parameter initialiser that sets the initial value of each element in a parameter tensor to a constant value.

gaussianInit
ParamInitializer gaussianInit(float mean, float stddev)

Creates a parameter initialiser that sets the initial value of each element in a parameter tensor to a different sample from a Gaussian distribution.

glorotGaussianInit
ParamInitializer glorotGaussianInit()

Creates a parameter initialiser that uses the method of Glorot and Bengio (2010).

glorotUniformInit
ParamInitializer glorotUniformInit()

Creates a parameter initialiser that uses the method of Glorot and Bengio (2010).

heGaussianInit
ParamInitializer heGaussianInit()

Creates a parameter initialiser that uses the method of He et al. (2015).

heUniformInit
ParamInitializer heUniformInit()

Creates a parameter initialiser that uses the method of He et al. (2015).

uniformInit
ParamInitializer uniformInit(float minval, float maxval)

Creates a parameter initialiser that sets the initial value of each element in a parameter tensor to a different sample from a uniform distribution.

Structs

Parameter
struct Parameter

Encapsulates information about network parameters.

Meta