Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
MegEngine 天元
MegEngine
提交
3ecded74
MegEngine
项目概览
MegEngine 天元
/
MegEngine
10 个月 前同步成功
通知
392
Star
4702
Fork
582
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
MegEngine
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
3ecded74
编写于
11月 25, 2020
作者:
M
Megvii Engine Team
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
refactor(distributed/server): use port 0 to get available port
GitOrigin-RevId: e367846b9216ea6d5ef7ada698ffba790ba8e1e6
上级
88e918e2
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
23 addition
and
12 deletion
+23
-12
imperative/python/megengine/distributed/launcher.py
imperative/python/megengine/distributed/launcher.py
+15
-6
imperative/python/megengine/distributed/server.py
imperative/python/megengine/distributed/server.py
+8
-6
未找到文件。
imperative/python/megengine/distributed/launcher.py
浏览文件 @
3ecded74
...
...
@@ -29,8 +29,8 @@ def launcher(func):
def
wrapper
(
*
args
,
**
kwargs
):
master_ip
=
"localhost"
port
=
get_free_ports
(
1
)[
0
]
server
=
Server
(
port
)
server
=
Server
()
port
=
server
.
py_server_port
procs
=
[]
for
rank
in
range
(
n_gpus
):
...
...
@@ -41,9 +41,18 @@ def launcher(func):
p
.
start
()
procs
.
append
(
p
)
for
rank
in
range
(
n_gpus
):
procs
[
rank
].
join
()
code
=
procs
[
rank
].
exitcode
assert
code
==
0
,
"subprocess {} exit with code {}"
.
format
(
rank
,
code
)
ranks
=
[
rank
for
rank
in
range
(
n_gpus
)]
while
len
(
ranks
)
>
0
:
left
=
[]
for
rank
in
ranks
:
procs
[
rank
].
join
(
1
)
code
=
procs
[
rank
].
exitcode
assert
(
code
==
0
or
code
==
None
),
"subprocess {} exit with code {}"
.
format
(
rank
,
code
)
if
code
==
None
:
left
.
append
(
rank
)
ranks
=
left
return
wrapper
imperative/python/megengine/distributed/server.py
浏览文件 @
3ecded74
...
...
@@ -10,6 +10,7 @@ import threading
import
time
from
collections
import
defaultdict
from
functools
import
partial
from
queue
import
Queue
from
socketserver
import
ThreadingMixIn
from
xmlrpc.client
import
ServerProxy
from
xmlrpc.server
import
SimpleXMLRPCServer
...
...
@@ -132,7 +133,7 @@ class ThreadXMLRPCServer(ThreadingMixIn, SimpleXMLRPCServer):
pass
def
start_server
(
py_server_port
,
mm_server_port
):
def
start_server
(
py_server_port
,
mm_server_port
,
queue
):
"""
Start python distributed server and multiple machine server.
...
...
@@ -141,6 +142,8 @@ def start_server(py_server_port, mm_server_port):
"""
server
=
ThreadXMLRPCServer
((
"0.0.0.0"
,
py_server_port
),
logRequests
=
False
)
server
.
register_instance
(
Methods
(
mm_server_port
))
_
,
port
=
server
.
server_address
queue
.
put
(
port
)
server
.
serve_forever
()
...
...
@@ -152,15 +155,14 @@ class Server:
:param port: python server port.
"""
def
__init__
(
self
,
port
):
self
.
py_server_port
=
get_free_ports
(
1
)[
0
]
if
port
==
0
else
port
def
__init__
(
self
,
port
=
0
):
self
.
mm_server_port
=
create_mm_server
(
"0.0.0.0"
,
0
)
q
=
Queue
()
self
.
proc
=
threading
.
Thread
(
target
=
start_server
,
args
=
(
self
.
py_server_port
,
self
.
mm_server_port
),
daemon
=
True
,
target
=
start_server
,
args
=
(
port
,
self
.
mm_server_port
,
q
),
daemon
=
True
,
)
self
.
proc
.
start
()
self
.
py_server_port
=
q
.
get
()
class
Client
:
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录