dlpy.model.NatGradSolver¶

class dlpy.model.NatGradSolver(approximation_type=1, learning_rate=0.001, learning_rate_policy='fixed', gamma=0.1, step_size=10, power=0.75, use_locking=True, clip_grad_max=None, clip_grad_min=None, steps=None, fcmp_learning_rate=None, lr_scheduler=None)¶

Bases: dlpy.model.Solver

Natural gradient solver object

Parameters:

approximation_type : int, optional Specifies the approximate natural gradient type. learning_rate : double, optional Specifies the learning rate for the deep learning algorithm. learning_rate_policy : string, optional Specifies the learning rate policy for the deep learning algorithm. Valid Values: FIXED, STEP, POLY, INV, MULTISTEP Default: FIXED gamma : double, optional Specifies the gamma for the learning rate policy. step_size : int, optional Specifies the step size when the learning rate policy is set to STEP. power : double, optional Specifies the power for the learning rate policy. use_locking : bool, optional When it is false, the gradients are computed asynchronously with multiple threads. clip_grad_max : double, optional Specifies the maximum gradient value. All gradients that are greater than the specified value are set to the specified value. clip_grad_min : double, optional Specifies the minimum gradient value. All gradients that are less than the specified value are set to the specified value. steps : list-of-ints, optional specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs. fcmp_learning_rate : string, optional specifies the FCMP learning rate function. lr_scheduler : LRScheduler, optional Specifies learning rate policy DLPy provides you with some predefined learning rate policies. 1. FixedLR 2. StepLR 3. MultiStepLR 4. PolynomialLR 5. ReduceLROnPlateau 6. CyclicLR Besides, you can also customize your own learning rate policy. You can find more examples at DLPy example folder.

Returns:

NatGradSolver

__init__(approximation_type=1, learning_rate=0.001, learning_rate_policy='fixed', gamma=0.1, step_size=10, power=0.75, use_locking=True, clip_grad_max=None, clip_grad_min=None, steps=None, fcmp_learning_rate=None, lr_scheduler=None)¶

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__([approximation_type, …])	Initialize self.
add_parameter(key, value)	Adds a parameter to the parameter list of a solver.
clear()
get(k[,d])
items()
keys()
pop(k[,d])	If key is not found, d is returned if given, otherwise KeyError is raised.
popitem()	as a 2-tuple; but raise KeyError if D is empty.
set_method(method)	Sets the solver method in the parameters list.
setdefault(k[,d])
update([E, ]**F)	If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
values()