Optimizer

class tinder.optimizer.WarmRestartLR(scheduler, T_mult=2)

Provide T_cur for the current restart session.

Example:

cosine = CosineAnnealingLR(..)
wr = WarmRestartLR(scheduler=cosine, T_mult=2)

for epoch in range(100):
    wr.step()
    ..

Parameters

• scheduler – instance of CosineAnnealingLR.

• T_mult (float) – ratio to increase T_cur. Default: 2.

tinder.optimizer.copy_opt_state(old: torch.optim.optimizer.Optimizer, new: torch.optim.optimizer.Optimizer)

Copy one optimizer’s state to another.

Parameters

• old (torch.optim.Optimizer) – A source optimizer

• new (torch.optim.Optimizer) – A destination optimizer