• J
    [sharding] doc, api, bug fixed (#28983) · 0dadacc4
    JZ-LIANG 提交于
    * add lars to fleet meta optimizer
    
    * add lamb to proto
    
    * add lamb to fleet meta optimizer
    
    * fixed syntax bug
    
    * fixed syntax bug
    
    * fixed syntax error in lamb, add config setter of lamb in distributed_strategy
    
    * trigger unitest to rerun
    
    * add new unitest func for lamb
    
    * revise unitest for lars and lamb
    
    * revise dgc meta unitest
    
    * revise lars document in distribute_strategy
    
    * revise lars lamb document in distributed_strategy.py
    
    * revise lars lamb document in distributed_strategy.py
    
    * add weight decay exclude logic to lars
    
    * restore optimzier.py
    
    * restore optimizer.py as develop except lars
    
    * add epsilon and exclude fn to distributed_sttrategy
    
    * add lars epsilon
    
    * revise unitest for fleet lars and lamb
    
    * revise lars lamb unitest for CI coverage
    
    * revise lars argument api
    
    * revise lars argument api
    
    * revise lars argument api
    
    * revise api doc of lars
    
    * fix op role
    
    * add sharding save and add_sync_comm_for_test function
    
    * add comm_analyse to utlis
    
    * revise sharding_utils
    
    * add sharding saving unittest
    
    * revise sharding utils for unittest
    
    * revise sharding en doc
    
    * update sharding utils api
    
    * add doc for sharding
    
    * fixed bug in sharding var size count
    
    * update varsize count in sharding
    
    * fix sharding num_nccl_comm
    
    * Revert "fix sharding num_nccl_comm"
    
    This reverts commit d51587c15e9323acf226ddd36154275f0d1daf76.
    0dadacc4
distributed_strategy.py 47.4 KB