- 03 8月, 2022 1 次提交
-
-
由 sneaxiy 提交于
* add use_hierarchical_allreduce * support hierarchical allreduce for more cases
-
- 02 7月, 2022 1 次提交
-
-
由 Leo Chen 提交于
* fix init() * delete test_device_context * replace CPUDeviceContext with CPUContext * fix test_scalar * remove dot_op.cc * fix compile
-
- 26 6月, 2022 1 次提交
-
-
由 Sing_chan 提交于
-
- 09 6月, 2022 1 次提交
-
-
由 sneaxiy 提交于
* add nproc_per_node for DistributedFusedLamb * fix nproc_per_node communicator bug * fix ring_id = 1 init bug * fix ci * fix test_parallel_executor_mnist.py
-
- 07 6月, 2022 1 次提交
-
-
由 sneaxiy 提交于
* add use_master_acc_grad * add ut
-
- 28 4月, 2022 1 次提交
-
-
由 sneaxiy 提交于
* add gradient merge for DistributedFusedLamb * use master acc gradient * fix CI ut * polish * remove math_function_impl.h change * fix test_update_loss_scaling_op.py * try to fix XPU/NPU CI * add gm ut
-
- 07 4月, 2022 1 次提交
-
-
由 sneaxiy 提交于
* add Output(Step) to distributed fused lamb op * add _set_step
-
- 01 3月, 2022 1 次提交
-
-
由 sneaxiy 提交于
* vectorize lamb kernel * remove flags, add ut * remove useless codes * refine code, add param order
-
- 25 2月, 2022 1 次提交
-
-
由 sneaxiy 提交于
* add multi tensor apply l2 norm * add multi_tensor_apply code * make sizeof(TensorMeta) smalller * move code to distributed_fused_lamb_op.cu * remove useless FLAGS
-
- 19 2月, 2022 1 次提交
-
-
由 sneaxiy 提交于
* add DistributedFusedLamb op * polish code * fix compile error * compatible with pten changement * fix rocm compile error * improve converage * update upstream/develop * fix cast_with_ptr.h * add FLAGS_distributed_lamb_divide_nranks_when_allreduce=1 * fix clip before allreduce * add use_master_param_norm * code polish * fix bug * fix ROCM ci
-