• J
    [Sharding] add new features (#28568) · 5a9f6889
    JZ-LIANG 提交于
    * add lars to fleet meta optimizer
    
    * add lamb to proto
    
    * add lamb to fleet meta optimizer
    
    * fixed syntax bug
    
    * fixed syntax bug
    
    * fixed syntax error in lamb, add config setter of lamb in distributed_strategy
    
    * trigger unitest to rerun
    
    * add new unitest func for lamb
    
    * revise unitest for lars and lamb
    
    * revise dgc meta unitest
    
    * revise lars document in distribute_strategy
    
    * revise lars lamb document in distributed_strategy.py
    
    * revise lars lamb document in distributed_strategy.py
    
    * add weight decay exclude logic to lars
    
    * restore optimzier.py
    
    * restore optimizer.py as develop except lars
    
    * add epsilon and exclude fn to distributed_sttrategy
    
    * add lars epsilon
    
    * revise unitest for fleet lars and lamb
    
    * revise lars lamb unitest for CI coverage
    
    * revise lars argument api
    
    * revise lars argument api
    
    * revise lars argument api
    
    * revise api doc of lars
    
    * fix op role
    
    * add sharding save and add_sync_comm_for_test function
    
    * add comm_analyse to utlis
    
    * revise sharding_utils
    
    * add sharding saving unittest
    
    * revise sharding utils for unittest
    5a9f6889
utils.py 13.9 KB