addding polynomial lr scheduler (#683)
Summary: Co-authored-by: jingfeidu <jingfeidu@fb.com> The implementation is by Jingfei Du from branch "bigbert". Copied over to this CR to get it merged in isolation since other changes seem to be already in master. **Small changes from original:** Added following line in `__init__` as discovered by myleott : ``` self.optimizer.set_lr(self.warmup_factor * self.lr) ``` Pull Request resolved: https://github.com/pytorch/fairseq/pull/683 Reviewed By: myleott Differential Revision: D15149628 Pulled By: myleott fbshipit-source-id: 5f715611182cdd111e636c66d5f24aa88fa03e29
Loading
Please register or sign in to comment