Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
MindSpore
mindinsight
提交
ed594114
M
mindinsight
项目概览
MindSpore
/
mindinsight
通知
8
Star
4
Fork
2
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
mindinsight
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
ed594114
编写于
6月 19, 2020
作者:
M
mindspore-ci-bot
提交者:
Gitee
6月 19, 2020
浏览文件
操作
浏览文件
下载
差异文件
!310 Fix ci problem.
Merge pull request !310 from yuximiao/master
上级
4de8ca61
ed6181ef
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
36 addition
and
26 deletion
+36
-26
mindinsight/profiler/analyser/minddata_analyser.py
mindinsight/profiler/analyser/minddata_analyser.py
+36
-26
未找到文件。
mindinsight/profiler/analyser/minddata_analyser.py
浏览文件 @
ed594114
...
...
@@ -73,11 +73,12 @@ class MinddataAnalyser(BaseAnalyser):
if
len
(
node_info
)
>
2
and
info_type
in
[
"all"
,
"time"
]:
time_info
[
"size"
]
=
len
(
time_list
)
time_info
[
"info"
]
=
{
"get_next"
:
time_list
}
time_info
[
"summary"
]
=
{
"time_summary"
:
{
"avg_cost"
:
"0"
if
not
time_list
else
str
(
total_cost
/
len
(
time_list
))
if
time_info
[
"size"
]:
time_info
[
"summary"
]
=
{
"time_summary"
:
{
"avg_cost"
:
"0"
if
not
time_list
else
str
(
total_cost
/
len
(
time_list
))
}
}
}
return
queue_info
,
time_info
...
...
@@ -103,15 +104,7 @@ class MinddataAnalyser(BaseAnalyser):
queue_size_list
=
[]
empty_step
,
full_step
=
0
,
0
device_queue_file_name
=
"device_queue_profiling_"
+
self
.
_device_id
+
".txt"
device_queue_file_path
=
MinddataAnalyser
.
find_target_file
(
self
.
_profiling_dir
,
device_queue_file_name
)
feed_file_name
=
"dataset_iterator_profiling_"
+
self
.
_device_id
+
".txt"
feed_file_path
=
MinddataAnalyser
.
find_target_file
(
self
.
_profiling_dir
,
feed_file_name
)
file_path
=
""
if
device_queue_file_path
:
file_path
=
device_queue_file_path
elif
not
device_queue_file_path
and
feed_file_path
:
file_path
=
feed_file_path
file_path
=
self
.
get_device_queue_file_path
()
if
file_path
:
with
open
(
file_path
)
as
data_file
:
...
...
@@ -145,9 +138,10 @@ class MinddataAnalyser(BaseAnalyser):
time_info
[
"info"
]
=
{
"total_cost"
:
total_time_list
,
"push_cost"
:
push_time_list
,
"get_cost"
:
get_time_list
}
time_info
[
"summary"
]
=
{
"time_summary"
:
{
"avg_cost"
:
total_cost
/
time_info
[
"size"
]}}
time_info
[
"summary"
][
"time_summary"
][
"get_cost"
]
=
total_get
/
time_info
[
"size"
]
time_info
[
"summary"
][
"time_summary"
][
"push_cost"
]
=
total_push
/
time_info
[
"size"
]
if
time_info
[
"size"
]:
time_info
[
"summary"
]
=
{
"time_summary"
:
{
"avg_cost"
:
total_cost
/
time_info
[
"size"
]}}
time_info
[
"summary"
][
"time_summary"
][
"get_cost"
]
=
total_get
/
time_info
[
"size"
]
time_info
[
"summary"
][
"time_summary"
][
"push_cost"
]
=
total_push
/
time_info
[
"size"
]
if
info_type
in
[
"all"
,
"queue"
]:
queue_size_list
=
MinddataAnalyser
.
sort_step
(
queue_size_list
)
...
...
@@ -159,6 +153,25 @@ class MinddataAnalyser(BaseAnalyser):
return
queue_info
,
time_info
def
get_device_queue_file_path
(
self
):
"""
Get device queue file path.
Returns:
str, the file path.
"""
device_queue_file_name
=
"device_queue_profiling_"
+
self
.
_device_id
+
".txt"
device_queue_file_path
=
MinddataAnalyser
.
find_target_file
(
self
.
_profiling_dir
,
device_queue_file_name
)
feed_file_name
=
"dataset_iterator_profiling_"
+
self
.
_device_id
+
".txt"
feed_file_path
=
MinddataAnalyser
.
find_target_file
(
self
.
_profiling_dir
,
feed_file_name
)
file_path
=
""
if
device_queue_file_path
:
file_path
=
device_queue_file_path
elif
not
device_queue_file_path
and
feed_file_path
:
file_path
=
feed_file_path
return
file_path
@
staticmethod
def
analyse_queue_summary
(
get_next_queue_info
,
device_queue_info
):
"""
...
...
@@ -171,6 +184,7 @@ class MinddataAnalyser(BaseAnalyser):
Returns:
dict, the summary of queue.
"""
result
=
{}
if
get_next_queue_info
and
device_queue_info
:
result
=
{
"data_process"
:
{
"status"
:
"normal"
},
"device_queue_op"
:
{
"status"
:
"normal"
},
...
...
@@ -190,13 +204,12 @@ class MinddataAnalyser(BaseAnalyser):
"summary"
,
{}).
get
(
"queue_summary"
,
{}).
get
(
"empty_queue"
,
0
)
device_queue_full_count
=
device_queue_info
.
get
(
"summary"
,
{}).
get
(
"queue_summary"
,
{}).
get
(
"full_queue"
,
0
)
result
[
"device_queue_info"
]
=
{
"summary"
:
{
"empty_batch_count"
:
device_queue_empty_count
,
"full_batch_count"
:
device_queue_full_count
,
"total_batch"
:
device_queue_info
.
get
(
"size"
)
}
}
result
[
"device_queue_info"
]
=
{
"summary"
:
{
"empty_batch_count"
:
device_queue_empty_count
,
"full_batch_count"
:
device_queue_full_count
,
"total_batch"
:
device_queue_info
.
get
(
"size"
)}}
if
get_next_queue_empty_count
:
if
device_queue_empty_count
>
device_queue_info
.
get
(
"size"
,
0
)
*
\
MinddataAnalyser
.
DEVICE_QUEUE_EMPTY_WARNING_THRESHOLD
:
...
...
@@ -226,9 +239,6 @@ class MinddataAnalyser(BaseAnalyser):
if
device_queue_empty_count
>
device_queue_info
.
get
(
"size"
,
0
)
*
0.7
:
result
[
"data_process"
][
"status"
]
=
"warning"
else
:
result
=
{}
return
result
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录