Add MuP optimizers (#2043)
* added paths for mup optimizers * added tests * formatting * Add license, fix missing distributed test, formatting * Add mpi4py to confirm tests work * Undo requirements change * Move to runtime folder * Rework to match new format * missing comma * hidden dim fix --------- Co-authored-by: NLogan Adams <114770087+loadams@users.noreply.github.com> Co-authored-by: NLogan Adams <loadams@microsoft.com>
Showing
想要评论请 注册 或 登录