Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
Crayon鑫
Paddle
提交
91bab752
P
Paddle
项目概览
Crayon鑫
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
未验证
提交
91bab752
编写于
11月 20, 2020
作者:
K
Kaipeng Deng
提交者:
GitHub
11月 20, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
fix dataloader default value and doc (#28728)
* fix dataloader. test=develop
上级
0ed80e09
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
11 addition
and
6 deletion
+11
-6
python/paddle/fluid/reader.py
python/paddle/fluid/reader.py
+5
-5
python/paddle/fluid/tests/unittests/test_multiprocess_dataloader_iterable_dataset_static.py
...s/test_multiprocess_dataloader_iterable_dataset_static.py
+2
-0
python/paddle/fluid/tests/unittests/test_multiprocess_dataloader_static.py
...id/tests/unittests/test_multiprocess_dataloader_static.py
+4
-1
未找到文件。
python/paddle/fluid/reader.py
浏览文件 @
91bab752
...
...
@@ -196,7 +196,7 @@ class DataLoader(object):
the key of the dict is the name of each fed variables. If
:attr:`return_list=True`, the return value on each device would
be a list(Tensor). :attr:`return_list` can only be True
in dynamic graph mode. Default
Fals
e.
in dynamic graph mode. Default
Tru
e.
batch_sampler(BatchSampler): an instance of `paddle.io.BatchSampler`
to generate batch indices to draw samples from :attr:`dataset`
and combine a batch. Default None.
...
...
@@ -308,7 +308,7 @@ class DataLoader(object):
dataset
,
feed_list
=
None
,
places
=
None
,
return_list
=
Fals
e
,
return_list
=
Tru
e
,
batch_sampler
=
None
,
batch_size
=
1
,
shuffle
=
False
,
...
...
@@ -403,10 +403,10 @@ class DataLoader(object):
if
self
.
dataset_kind
==
_DatasetKind
.
ITER
:
raise
ValueError
(
"length of IterableDataset not supported"
)
else
:
if
self
.
batch_size
is
None
:
return
len
(
self
.
dataset
)
else
:
if
self
.
auto_collate_batch
:
return
len
(
self
.
batch_sampler
)
else
:
return
len
(
self
.
dataset
)
def
__iter__
(
self
):
if
self
.
num_workers
==
0
:
...
...
python/paddle/fluid/tests/unittests/test_multiprocess_dataloader_iterable_dataset_static.py
浏览文件 @
91bab752
...
...
@@ -112,6 +112,7 @@ class TestStaticDataLoader(unittest.TestCase):
places
=
places
,
num_workers
=
num_workers
,
batch_size
=
BATCH_SIZE
,
return_list
=
False
,
drop_last
=
True
)
# assert len(dataloader) == int(SAMPLE_NUM / BATCH_SIZE)
...
...
@@ -199,6 +200,7 @@ class TestStaticDataLoaderWithBatchedDataset(TestStaticDataLoader):
places
=
places
,
num_workers
=
num_workers
,
batch_size
=
None
,
return_list
=
False
,
drop_last
=
True
)
exe
=
fluid
.
Executor
(
place
=
places
[
0
])
...
...
python/paddle/fluid/tests/unittests/test_multiprocess_dataloader_static.py
浏览文件 @
91bab752
...
...
@@ -113,6 +113,7 @@ class TestStaticDataLoader(unittest.TestCase):
places
=
places
,
num_workers
=
num_workers
,
batch_size
=
BATCH_SIZE
,
return_list
=
False
,
drop_last
=
True
)
assert
len
(
dataloader
)
==
int
(
SAMPLE_NUM
/
BATCH_SIZE
)
...
...
@@ -226,7 +227,8 @@ class RandomBatchedDataset(Dataset):
labels
=
[]
for
_
in
range
(
BATCH_SIZE
):
image
=
np
.
random
.
random
([
IMAGE_SIZE
]).
astype
(
'float32'
)
label
=
np
.
random
.
randint
(
0
,
self
.
class_num
-
1
,
(
1
,
)).
astype
(
'int64'
)
label
=
np
.
random
.
randint
(
0
,
self
.
class_num
-
1
,
(
1
,
)).
astype
(
'int64'
)
images
.
append
(
image
)
labels
.
append
(
label
)
return
np
.
stack
(
images
,
axis
=
0
),
np
.
stack
(
labels
,
axis
=
0
)
...
...
@@ -248,6 +250,7 @@ class TestStaticDataLoaderWithBatchedDataset(TestStaticDataLoader):
places
=
places
,
num_workers
=
num_workers
,
batch_size
=
None
,
return_list
=
False
,
drop_last
=
True
)
assert
len
(
dataloader
)
==
int
(
SAMPLE_NUM
/
BATCH_SIZE
)
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录