Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
ujs_wantz
Paddle
提交
817f9ef0
P
Paddle
项目概览
ujs_wantz
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
817f9ef0
编写于
10月 13, 2021
作者:
C
caozhou
提交者:
GitHub
10月 13, 2021
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
fix pp comm init bug (#36377)
上级
192e08cb
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
7 addition
and
1 deletion
+7
-1
python/paddle/distributed/auto_parallel/reshard.py
python/paddle/distributed/auto_parallel/reshard.py
+4
-1
python/paddle/fluid/tests/unittests/test_auto_parallel_reshard.py
...addle/fluid/tests/unittests/test_auto_parallel_reshard.py
+3
-0
未找到文件。
python/paddle/distributed/auto_parallel/reshard.py
浏览文件 @
817f9ef0
...
@@ -662,7 +662,10 @@ def _concat_partitions_with_op(partition_tensor_list, tensor, partition_index,
...
@@ -662,7 +662,10 @@ def _concat_partitions_with_op(partition_tensor_list, tensor, partition_index,
def
_init_comm_for_send_recv
():
def
_init_comm_for_send_recv
():
if
not
PROCESS_GROUP_MAP
[
"global_group"
].
is_instantiate
():
if
not
PROCESS_GROUP_MAP
:
genv
=
_get_global_env
()
PROCESS_GROUP_MAP
[
"global_group"
]
=
ProcessGroup
(
0
,
list
(
range
(
genv
.
world_size
)))
PROCESS_GROUP_MAP
[
"global_group"
].
instantiate
()
PROCESS_GROUP_MAP
[
"global_group"
].
instantiate
()
...
...
python/paddle/fluid/tests/unittests/test_auto_parallel_reshard.py
浏览文件 @
817f9ef0
...
@@ -27,6 +27,7 @@ from paddle.distributed import fleet
...
@@ -27,6 +27,7 @@ from paddle.distributed import fleet
from
paddle.distributed.auto_parallel.partitioner
import
Partitioner
from
paddle.distributed.auto_parallel.partitioner
import
Partitioner
from
paddle.distributed.auto_parallel.completion
import
complete_backward_annotation
from
paddle.distributed.auto_parallel.completion
import
complete_backward_annotation
from
paddle.distributed.auto_parallel.reshard
import
reshard
from
paddle.distributed.auto_parallel.reshard
import
reshard
from
paddle.distributed.auto_parallel.process
import
PROCESS_GROUP_MAP
paddle
.
enable_static
()
paddle
.
enable_static
()
_global_parallel_strategy
=
None
_global_parallel_strategy
=
None
...
@@ -254,6 +255,8 @@ class TestMLPReshard(unittest.TestCase):
...
@@ -254,6 +255,8 @@ class TestMLPReshard(unittest.TestCase):
dist_main_prog
,
dist_startup_prog
=
get_dist_prog
(
dist_main_prog
,
dist_startup_prog
=
get_dist_prog
(
train_program
,
startup_program
,
dist_context
,
rank_id
)
train_program
,
startup_program
,
dist_context
,
rank_id
)
complete_backward_annotation
(
dist_main_prog
,
dist_context
)
complete_backward_annotation
(
dist_main_prog
,
dist_context
)
for
key
in
list
(
PROCESS_GROUP_MAP
.
keys
()):
del
PROCESS_GROUP_MAP
[
key
]
reshard
(
dist_main_prog
,
dist_startup_prog
,
rank_id
,
dist_context
)
reshard
(
dist_main_prog
,
dist_startup_prog
,
rank_id
,
dist_context
)
# check send and recv result
# check send and recv result
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录