I am aware that the BatchNormalization layer is already much discussed in the Keras docs because of its different behavior during training and inference. However, the fact that the "gamma" parameter of my BatchNormalization layers updated even if I set the "training" argument to False when calling them surprises me a bit. After reading the docs I thought to have understood that it is important to freeze these layers completely when finetuning models and that (for the special case of BatchNormalization Layers) this can be achieved by setting the "training" argument to False when calling the layers. While this special behavior proved true for the "moving_mean" and "moving_variance" parameters of the layers it did not for the "gamma" parameter. Is this intended?