class Iterable[Tuple[str, torch.Tensor]], decay: float = 0.9999, numerator: float = 1.0, denominator: float = 10.0)[source]


Create shadow variables and maintain exponential moving average for model parameters.

parameters : Iterable[Tuple[str, Parameter]], required

The parameters whose averages we’ll be tracking.

decay : float, optional (default = 0.9999)

The decay rate that will be used if num_updates is not passed (and that will be used as an upper bound if num_updates is passed).

numerator : float, optional (default = 1.0)

The numerator used to compute the decay rate if num_updates is passed.

denominator : float, optional (default = 10.0)

The denominator used to compute the decay rate if num_updates is passed.

apply(num_updates: Optional[int] = None) → None[source]

Apply exponential moving average to named_parameters if specified, or we will apply this to all the trainable parameters of the model.

The optional num_updates parameter allows one to tweak the decay rate dynamically. If passed, the actual decay rate used is:

min(decay, (numerator + num_updates) / (denominator + num_updates))
(This logic is based on the Tensorflow exponential moving average
class Iterable[Tuple[str, torch.Tensor]])[source]

Bases: allennlp.common.registrable.Registrable

Tracks a moving average of model parameters.

apply(num_updates: Optional[int] = None)[source]

Update the moving averages based on the latest values of the parameters.

assign_average_value() → None[source]

Replace all the parameter values with the averages. Save the current parameter values to restore later.

default_implementation = 'exponential'
restore() → None[source]

Restore the backed-up (non-average) parameter values.