Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
机器未来
Paddle
提交
cf0a0579
P
Paddle
项目概览
机器未来
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
cf0a0579
编写于
1月 21, 2019
作者:
Q
Qiao Longfei
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
add document for ctr reader
test=develop
上级
45578c1b
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
29 addition
and
5 deletion
+29
-5
python/paddle/fluid/contrib/reader/README.md
python/paddle/fluid/contrib/reader/README.md
+15
-0
python/paddle/fluid/contrib/reader/ctr_reader.py
python/paddle/fluid/contrib/reader/ctr_reader.py
+14
-5
未找到文件。
python/paddle/fluid/contrib/reader/README.md
0 → 100644
浏览文件 @
cf0a0579
## CTR READER
An multi-thread cpp reader that has the same interface with py_reader. It
uses cpp multi-thread to read file and is much more faster then the Python read
thread in py_reader.
Currently, it support two types of file:
-
gzip
-
plain text file
and two types of data format:
-
cvs data format is :
*
label dense_fea,dense_fea sparse_fea,sparse_fea
-
the svm data format is :
*
label slot1:fea_sign slot2:fea_sign slot1:fea_sign
python/paddle/fluid/contrib/reader/ctr_reader.py
浏览文件 @
cf0a0579
...
@@ -54,8 +54,8 @@ def ctr_reader(
...
@@ -54,8 +54,8 @@ def ctr_reader(
feed_dict
,
feed_dict
,
file_type
,
# gzip or plain
file_type
,
# gzip or plain
file_format
,
# csv or svm
file_format
,
# csv or svm
dense_slot_index
s
,
dense_slot_index
,
sparse_slot_index
s
,
sparse_slot_index
,
capacity
,
capacity
,
thread_num
,
thread_num
,
batch_size
,
batch_size
,
...
@@ -78,11 +78,20 @@ def ctr_reader(
...
@@ -78,11 +78,20 @@ def ctr_reader(
Note that :code:`Program.clone()` method cannot clone :code:`py_reader`.
Note that :code:`Program.clone()` method cannot clone :code:`py_reader`.
Args:
Args:
feed_dict(list(variable)): a list of data variable.
file_type('gzip'|'plain'): the type of the data file
file_format('csv'|'svm'): csv data or svm data format.
cvs data format is :
label dense_fea,dense_fea sparse_fea,sparse_fea
the svm data format is :
label slot1:fea_sign slot2:fea_sign slot1:fea_sign
dense_slot_index(list(int)): the index of dense slots
sparse_slot_index(list(int)): the index of sparse slots
capacity(int): The buffer capacity maintained by :code:`py_reader`.
capacity(int): The buffer capacity maintained by :code:`py_reader`.
thread_num(list|tuple): List of tuples which declaring data shapes.
thread_num(list|tuple): List of tuples which declaring data shapes.
batch_size(list|tuple): List of strs which declaring data type.
batch_size(list|tuple): List of strs which declaring data type.
file_list(list|tuple): List of ints which declaring data lod_level.
file_list(list|tuple): List of ints which declaring data lod_level.
slots(bool):
Whether use double buffer or not.
slots(bool):
slot id of all sparse feature
name(basestring): The prefix Python queue name and Reader name. None will
name(basestring): The prefix Python queue name and Reader name. None will
be generated automatically.
be generated automatically.
...
@@ -116,8 +125,8 @@ def ctr_reader(
...
@@ -116,8 +125,8 @@ def ctr_reader(
'file_list'
:
file_list
,
'file_list'
:
file_list
,
'file_type'
:
file_type
,
'file_type'
:
file_type
,
'file_format'
:
file_format
,
'file_format'
:
file_format
,
'dense_slot_index'
:
dense_slot_index
s
,
'dense_slot_index'
:
dense_slot_index
,
'sparse_slot_index'
:
sparse_slot_index
s
,
'sparse_slot_index'
:
sparse_slot_index
,
'sparse_slots'
:
slots
,
'sparse_slots'
:
slots
,
'ranks'
:
[],
'ranks'
:
[],
'lod_levels'
:
[],
'lod_levels'
:
[],
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录