提交 92c2c0ce 编写于 作者: S shippingwang

fix dist strategy

上级 ad6b6deb
...@@ -8,6 +8,8 @@ ARCHITECTURE: ...@@ -8,6 +8,8 @@ ARCHITECTURE:
drop_connect_rate: 0.1 drop_connect_rate: 0.1
fix_head_stem: True fix_head_stem: True
relu_fn: True relu_fn: True
local_pooling: True
use_se: False
pretrained_model: "" pretrained_model: ""
model_save_dir: "./output/" model_save_dir: "./output/"
......
...@@ -297,6 +297,8 @@ def dist_optimizer(config, optimizer): ...@@ -297,6 +297,8 @@ def dist_optimizer(config, optimizer):
dist_strategy.nccl_comm_num = 1 dist_strategy.nccl_comm_num = 1
dist_strategy.fuse_all_reduce_ops = True dist_strategy.fuse_all_reduce_ops = True
dist_strategy.exec_strategy = exec_strategy dist_strategy.exec_strategy = exec_strategy
dist_strategy.mode = "collective"
dist_strategy.collective_mode = "grad_allreduce"
optimizer = fleet.distributed_optimizer(optimizer, strategy=dist_strategy) optimizer = fleet.distributed_optimizer(optimizer, strategy=dist_strategy)
return optimizer return optimizer
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册