Rewrite Adam and LazyAdam optimizer to take global step for computing beta1...
Rewrite Adam and LazyAdam optimizer to take global step for computing beta1 and beta2 accumulators, instead of having the optimizer instance to keep its own independent beta1 and beta2 accumulators as non-slot variables. PiperOrigin-RevId: 224948020
Showing
想要评论请 注册 或 登录