Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
MindSpore
mindspore
提交
c415e8ce
M
mindspore
项目概览
MindSpore
/
mindspore
通知
35
Star
15
Fork
15
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
mindspore
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
c415e8ce
编写于
9月 08, 2020
作者:
M
mindspore-ci-bot
提交者:
Gitee
9月 08, 2020
浏览文件
操作
浏览文件
下载
差异文件
!5860 Fix dataset register occational problem
Merge pull request !5860 from xiefangqi/fix_dataset_register_issue
上级
0d7bca08
9b8c19d9
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
12 addition
and
4 deletion
+12
-4
mindspore/ccsrc/minddata/dataset/engine/datasetops/source/clue_op.cc
...csrc/minddata/dataset/engine/datasetops/source/clue_op.cc
+4
-1
mindspore/ccsrc/minddata/dataset/engine/datasetops/source/csv_op.cc
...ccsrc/minddata/dataset/engine/datasetops/source/csv_op.cc
+4
-1
mindspore/ccsrc/minddata/dataset/engine/datasetops/source/text_file_op.cc
...minddata/dataset/engine/datasetops/source/text_file_op.cc
+4
-2
未找到文件。
mindspore/ccsrc/minddata/dataset/engine/datasetops/source/clue_op.cc
浏览文件 @
c415e8ce
...
...
@@ -221,6 +221,10 @@ Status ClueOp::LoadFile(const std::string &file, const int64_t start_offset, con
Status
ClueOp
::
operator
()()
{
RETURN_IF_NOT_OK
(
CalculateNumRowsPerShard
());
// Move register to the front of launching thread, this will fix the problem
// when thread exit unnormally register will failed occasionally.
RETURN_IF_NOT_OK
(
io_block_queue_wait_post_
.
Register
(
tree_
->
AllTasks
()));
// launch one thread, responsible for filling IoBlockQueue
RETURN_IF_NOT_OK
(
tree_
->
LaunchWorkers
(
1
,
std
::
bind
(
&
ClueOp
::
WaitToFillIOBlockQueue
,
this
)));
...
...
@@ -228,7 +232,6 @@ Status ClueOp::operator()() {
// must be called after launching workers.
TaskManager
::
FindMe
()
->
Post
();
RETURN_IF_NOT_OK
(
io_block_queue_wait_post_
.
Register
(
tree_
->
AllTasks
()));
NotifyToFillIOBlockQueue
();
while
(
!
finished_reading_dataset_
)
{
...
...
mindspore/ccsrc/minddata/dataset/engine/datasetops/source/csv_op.cc
浏览文件 @
c415e8ce
...
...
@@ -518,6 +518,10 @@ Status CsvOp::LoadFile(const std::string &file, const int64_t start_offset, cons
Status
CsvOp
::
operator
()()
{
RETURN_IF_NOT_OK
(
CalculateNumRowsPerShard
());
// Move register to the front of launching thread, this will fix the problem
// when thread exit unnormally register will failed occasionally.
RETURN_IF_NOT_OK
(
io_block_queue_wait_post_
.
Register
(
tree_
->
AllTasks
()));
// launch one thread, responsible for filling IoBlockQueue
RETURN_IF_NOT_OK
(
tree_
->
LaunchWorkers
(
1
,
std
::
bind
(
&
CsvOp
::
WaitToFillIOBlockQueue
,
this
)));
...
...
@@ -525,7 +529,6 @@ Status CsvOp::operator()() {
// must be called after launching workers.
TaskManager
::
FindMe
()
->
Post
();
RETURN_IF_NOT_OK
(
io_block_queue_wait_post_
.
Register
(
tree_
->
AllTasks
()));
NotifyToFillIOBlockQueue
();
while
(
!
finished_reading_dataset_
)
{
...
...
mindspore/ccsrc/minddata/dataset/engine/datasetops/source/text_file_op.cc
浏览文件 @
c415e8ce
...
...
@@ -378,6 +378,10 @@ void TextFileOp::NotifyToFillIOBlockQueue() { io_block_queue_wait_post_.Set(); }
Status
TextFileOp
::
operator
()()
{
RETURN_IF_NOT_OK
(
CalculateNumRowsPerShard
());
// Move register to the front of launching thread, this will fix the problem
// when thread exit unnormally register will failed occasionally.
RETURN_IF_NOT_OK
(
io_block_queue_wait_post_
.
Register
(
tree_
->
AllTasks
()));
// launch one thread, responsible for filling IoBlockQueue
RETURN_IF_NOT_OK
(
tree_
->
LaunchWorkers
(
1
,
std
::
bind
(
&
TextFileOp
::
WaitToFillIOBlockQueue
,
this
)));
...
...
@@ -387,8 +391,6 @@ Status TextFileOp::operator()() {
// must be called after launching workers.
TaskManager
::
FindMe
()
->
Post
();
RETURN_IF_NOT_OK
(
io_block_queue_wait_post_
.
Register
(
tree_
->
AllTasks
()));
NotifyToFillIOBlockQueue
();
while
(
!
finished_reading_dataset_
)
{
int64_t
buffer_id
=
0
;
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录