Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
PaddlePaddle
Paddle
提交
34410eb8
P
Paddle
项目概览
PaddlePaddle
/
Paddle
10 个月 前同步成功
通知
2285
Star
20931
Fork
5422
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1423
列表
看板
标记
里程碑
合并请求
543
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1,423
Issue
1,423
列表
看板
标记
里程碑
合并请求
543
合并请求
543
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
34410eb8
编写于
11月 08, 2017
作者:
C
caoying03
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
nce does not need activation.
上级
d9e5eba0
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
35 addition
and
31 deletion
+35
-31
python/paddle/trainer_config_helpers/layers.py
python/paddle/trainer_config_helpers/layers.py
+35
-31
未找到文件。
python/paddle/trainer_config_helpers/layers.py
浏览文件 @
34410eb8
...
@@ -5494,7 +5494,11 @@ def crf_decoding_layer(input,
...
@@ -5494,7 +5494,11 @@ def crf_decoding_layer(input,
return
LayerOutput
(
name
,
LayerType
.
CRF_DECODING_LAYER
,
parents
,
size
=
1
)
return
LayerOutput
(
name
,
LayerType
.
CRF_DECODING_LAYER
,
parents
,
size
=
1
)
@
wrap_act_default
(
act
=
SigmoidActivation
())
"""
Following are cost Layers.
"""
@
wrap_bias_attr_default
(
has_bias
=
True
)
@
wrap_bias_attr_default
(
has_bias
=
True
)
@
wrap_param_attr_default
()
@
wrap_param_attr_default
()
@
wrap_name_default
()
@
wrap_name_default
()
...
@@ -5502,7 +5506,6 @@ def crf_decoding_layer(input,
...
@@ -5502,7 +5506,6 @@ def crf_decoding_layer(input,
def
nce_layer
(
input
,
def
nce_layer
(
input
,
label
,
label
,
num_classes
=
None
,
num_classes
=
None
,
act
=
None
,
param_attr
=
None
,
param_attr
=
None
,
weight
=
None
,
weight
=
None
,
num_neg_samples
=
10
,
num_neg_samples
=
10
,
...
@@ -5511,9 +5514,12 @@ def nce_layer(input,
...
@@ -5511,9 +5514,12 @@ def nce_layer(input,
bias_attr
=
None
,
bias_attr
=
None
,
layer_attr
=
None
):
layer_attr
=
None
):
"""
"""
Noise-contrastive estimation.
Noise-contrastive estimation. This layer implements the method in the
Implements the method in the following paper:
following paper:
A fast and simple algorithm for training neural probabilistic language models.
Reference:
A fast and simple algorithm for training neural probabilistic language
models. https://www.cs.toronto.edu/~amnih/papers/ncelm.pdf
The example usage is:
The example usage is:
...
@@ -5525,32 +5531,37 @@ def nce_layer(input,
...
@@ -5525,32 +5531,37 @@ def nce_layer(input,
:param name: The name of this layer. It is optional.
:param name: The name of this layer. It is optional.
:type name: basestring
:type name: basestring
:param input: The input layers. It could be a LayerOutput of list/tuple of LayerOutput.
:param input: The input layers. It should be a LayerOutput or a list/tuple
of LayerOutput.
:type input: LayerOutput | list | tuple | collections.Sequence
:type input: LayerOutput | list | tuple | collections.Sequence
:param label:
label layer
:param label:
The ground truth.
:type label: LayerOutput
:type label: LayerOutput
:param weight: weight layer, can be None(default)
:param weight: The weight layer defines a weight for each sample in the
mini-batch. The default value is None.
:type weight: LayerOutput
:type weight: LayerOutput
:param num_classes:
number of classes
.
:param num_classes:
The class number
.
:type num_classes: int
:type num_classes: int
:param act: Activation type. SigmoidActivation is the default.
:param param_attr: The parameter attributes.
:type act: BaseActivation
:type param_attr: ParameterAttribute|list
:param param_attr: The Parameter Attribute|list.
:param num_neg_samples: The number of sampled negative labels. The default
:type param_attr: ParameterAttribute
value is 10.
:param num_neg_samples: number of negative samples. Default is 10.
:type num_neg_samples: int
:type num_neg_samples: int
:param neg_distribution: The distribution for generating the random negative labels.
:param neg_distribution: The discrete noisy distribution over the output
A uniform distribution will be used if not provided.
space from which num_neg_samples negative labels
If not None, its length must be equal to num_classes.
are sampled. If this parameter is not set, a
uniform distribution will be used. A user defined
distribution is a list whose length must be equal
to the num_classes. Each member of the list defines
the probability of a class given input x.
:type neg_distribution: list | tuple | collections.Sequence | None
:type neg_distribution: list | tuple | collections.Sequence | None
:param bias_attr: The
Bias Attribute. If the parameter is set to
:param bias_attr: The
attribute for bias. If this parameter is set False or
False or something not type of ParameterAttribute,
any object whose type is not ParameterAttribute, no bias
no bias is defined. If the parameter is set to
is added. If this parameter is set True, the bias is
True, the bias is
initialized to zero.
initialized to zero.
:type bias_attr: ParameterAttribute | None | bool | Any
:type bias_attr: ParameterAttribute | None | bool | Any
:param layer_attr: Extra Layer Attribute.
:param layer_attr: Extra Layer Attribute.
:type layer_attr: ExtraLayerAttribute
:type layer_attr: ExtraLayerAttribute
:return:
layer name
.
:return:
The LayerOutput object
.
:rtype: LayerOutput
:rtype: LayerOutput
"""
"""
if
isinstance
(
input
,
LayerOutput
):
if
isinstance
(
input
,
LayerOutput
):
...
@@ -5573,8 +5584,6 @@ def nce_layer(input,
...
@@ -5573,8 +5584,6 @@ def nce_layer(input,
assert
isinstance
(
neg_distribution
,
collections
.
Sequence
)
assert
isinstance
(
neg_distribution
,
collections
.
Sequence
)
assert
len
(
neg_distribution
)
==
num_classes
assert
len
(
neg_distribution
)
==
num_classes
assert
abs
(
sum
(
neg_distribution
)
-
1.0
)
<
1e-5
assert
abs
(
sum
(
neg_distribution
)
-
1.0
)
<
1e-5
if
not
isinstance
(
act
,
BaseActivation
):
raise
TypeError
()
ipts_for_layer
=
[]
ipts_for_layer
=
[]
parents
=
[]
parents
=
[]
...
@@ -5596,7 +5605,7 @@ def nce_layer(input,
...
@@ -5596,7 +5605,7 @@ def nce_layer(input,
type
=
LayerType
.
NCE_LAYER
,
type
=
LayerType
.
NCE_LAYER
,
num_classes
=
num_classes
,
num_classes
=
num_classes
,
neg_sampling_dist
=
neg_distribution
,
neg_sampling_dist
=
neg_distribution
,
active_type
=
act
.
name
,
active_type
=
SigmoidActivation
()
.
name
,
num_neg_samples
=
num_neg_samples
,
num_neg_samples
=
num_neg_samples
,
inputs
=
ipts_for_layer
,
inputs
=
ipts_for_layer
,
bias
=
ParamAttr
.
to_bias
(
bias_attr
),
bias
=
ParamAttr
.
to_bias
(
bias_attr
),
...
@@ -5606,12 +5615,7 @@ def nce_layer(input,
...
@@ -5606,12 +5615,7 @@ def nce_layer(input,
LayerType
.
NCE_LAYER
,
LayerType
.
NCE_LAYER
,
parents
=
parents
,
parents
=
parents
,
size
=
l
.
config
.
size
,
size
=
l
.
config
.
size
,
activation
=
act
)
activation
=
SigmoidActivation
())
"""
following are cost Layers.
"""
@
wrap_name_default
()
@
wrap_name_default
()
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录