Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
正统之独孤求败
mindspore
提交
447420eb
M
mindspore
项目概览
正统之独孤求败
/
mindspore
与 Fork 源项目一致
Fork自
MindSpore / mindspore
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
mindspore
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
447420eb
编写于
5月 22, 2020
作者:
L
liyong
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
fix bug when append data by absolute path
上级
d9c74e0a
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
31 addition
and
1 deletion
+31
-1
mindspore/ccsrc/mindrecord/io/shard_writer.cc
mindspore/ccsrc/mindrecord/io/shard_writer.cc
+1
-1
tests/ut/python/mindrecord/test_mindrecord_base.py
tests/ut/python/mindrecord/test_mindrecord_base.py
+30
-0
未找到文件。
mindspore/ccsrc/mindrecord/io/shard_writer.cc
浏览文件 @
447420eb
...
...
@@ -201,7 +201,7 @@ MSRStatus ShardWriter::OpenForAppend(const std::string &path) {
if
(
ret
==
FAILED
)
{
return
FAILED
;
}
ret
=
Open
(
json_header
[
"shard_addresses"
]
,
true
);
ret
=
Open
(
real_addresses
,
true
);
if
(
ret
==
FAILED
)
{
MS_LOG
(
ERROR
)
<<
"Open file failed"
;
return
FAILED
;
...
...
tests/ut/python/mindrecord/test_mindrecord_base.py
浏览文件 @
447420eb
...
...
@@ -25,6 +25,7 @@ FILES_NUM = 4
CV_FILE_NAME
=
"./imagenet.mindrecord"
CV2_FILE_NAME
=
"./imagenet_loop.mindrecord"
CV3_FILE_NAME
=
"./imagenet_append.mindrecord"
CV4_FILE_NAME
=
"/tmp/imagenet_append.mindrecord"
NLP_FILE_NAME
=
"./aclImdb.mindrecord"
...
...
@@ -170,6 +171,35 @@ def test_cv_file_append_writer():
os
.
remove
(
"{}.db"
.
format
(
x
))
def
test_cv_file_append_writer_absolute_path
():
"""tutorial for cv dataset append writer."""
writer
=
FileWriter
(
CV4_FILE_NAME
,
4
)
data
=
get_data
(
"../data/mindrecord/testImageNetData/"
)
cv_schema_json
=
{
"file_name"
:
{
"type"
:
"string"
},
"label"
:
{
"type"
:
"int64"
},
"data"
:
{
"type"
:
"bytes"
}}
writer
.
add_schema
(
cv_schema_json
,
"img_schema"
)
writer
.
add_index
([
"file_name"
,
"label"
])
writer
.
write_raw_data
(
data
[
0
:
5
])
writer
.
commit
()
write_append
=
FileWriter
.
open_for_append
(
CV4_FILE_NAME
+
"0"
)
write_append
.
write_raw_data
(
data
[
5
:
10
])
write_append
.
commit
()
reader
=
FileReader
(
CV4_FILE_NAME
+
"0"
)
count
=
0
for
index
,
x
in
enumerate
(
reader
.
get_next
()):
assert
len
(
x
)
==
3
count
=
count
+
1
logger
.
info
(
"#item{}: {}"
.
format
(
index
,
x
))
assert
count
==
10
reader
.
close
()
paths
=
[
"{}{}"
.
format
(
CV4_FILE_NAME
,
str
(
x
).
rjust
(
1
,
'0'
))
for
x
in
range
(
4
)]
for
x
in
paths
:
os
.
remove
(
"{}"
.
format
(
x
))
os
.
remove
(
"{}.db"
.
format
(
x
))
def
test_cv_file_writer_loop_and_read
():
"""tutorial for cv dataset loop writer."""
writer
=
FileWriter
(
CV2_FILE_NAME
,
FILES_NUM
)
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录