未验证 提交 e6654c1c 编写于 作者: Y Yu Yang 提交者: GitHub

Merge pull request #11489 from wanghaoshuang/whs_doc1

 Fix doc of warpctc, array_read, edit_distance and sequence_reshape.
...@@ -1055,19 +1055,38 @@ def equal(x, y, cond=None, **ignored): ...@@ -1055,19 +1055,38 @@ def equal(x, y, cond=None, **ignored):
def array_read(array, i): def array_read(array, i):
"""This function performs the operation to read the data in as an """
This function performs the operation to read the data in as an
LOD_TENSOR_ARRAY. LOD_TENSOR_ARRAY.
.. code-block:: text
Given:
array = [0.6, 0.1, 0.3, 0.1]
And:
i = 2
Then:
output = 0.3
Args: Args:
array (Variable|list): The input tensor that will be written to an array. array (Variable|list): The input tensor that store data to be read.
i (Variable|list): The subscript index in tensor array, that points the i (Variable|list): The index of the data to be read from input array.
place where data will be written to.
Returns: Returns:
Variable: The tensor type variable that has the data written to it. Variable: The tensor type variable that has the data written to it.
Examples: Examples:
.. code-block::python .. code-block:: python
tmp = fluid.layers.zeros(shape=[10], dtype='int32')
i = fluid.layers.fill_constant(shape=[1], dtype='int64', value=10) tmp = fluid.layers.zeros(shape=[10], dtype='int32')
arr = layers.array_read(tmp, i=i) i = fluid.layers.fill_constant(shape=[1], dtype='int64', value=10)
arr = fluid.layers.array_read(tmp, i=i)
""" """
helper = LayerHelper('array_read', **locals()) helper = LayerHelper('array_read', **locals())
if not isinstance( if not isinstance(
......
...@@ -25,21 +25,74 @@ import utils ...@@ -25,21 +25,74 @@ import utils
import random import random
__all__ = [ __all__ = [
'fc', 'embedding', 'dynamic_lstm', 'dynamic_lstmp', 'dynamic_gru', 'fc',
'gru_unit', 'linear_chain_crf', 'crf_decoding', 'cos_sim', 'cross_entropy', 'embedding',
'square_error_cost', 'chunk_eval', 'sequence_conv', 'conv2d', 'conv3d', 'dynamic_lstm',
'sequence_pool', 'sequence_softmax', 'softmax', 'pool2d', 'pool3d', 'dynamic_lstmp',
'batch_norm', 'beam_search_decode', 'conv2d_transpose', 'conv3d_transpose', 'dynamic_gru',
'sequence_expand', 'lstm_unit', 'reduce_sum', 'reduce_mean', 'reduce_max', 'gru_unit',
'reduce_min', 'reduce_prod', 'sequence_first_step', 'sequence_last_step', 'linear_chain_crf',
'dropout', 'split', 'ctc_greedy_decoder', 'edit_distance', 'l2_normalize', 'crf_decoding',
'matmul', 'topk', 'warpctc', 'sequence_reshape', 'transpose', 'im2sequence', 'cos_sim',
'nce', 'beam_search', 'row_conv', 'multiplex', 'layer_norm', 'cross_entropy',
'softmax_with_cross_entropy', 'smooth_l1', 'one_hot', 'square_error_cost',
'autoincreased_step_counter', 'reshape', 'lod_reset', 'lrn', 'pad', 'chunk_eval',
'label_smooth', 'roi_pool', 'dice_loss', 'image_resize', 'sequence_conv',
'image_resize_short', 'resize_bilinear', 'gather', 'random_crop', 'conv2d',
'mean_iou', 'relu', 'log' 'conv3d',
'sequence_pool',
'sequence_softmax',
'softmax',
'pool2d',
'pool3d',
'batch_norm',
'beam_search_decode',
'conv2d_transpose',
'conv3d_transpose',
'sequence_expand',
'lstm_unit',
'reduce_sum',
'reduce_mean',
'reduce_max',
'reduce_min',
'reduce_prod',
'sequence_first_step',
'sequence_last_step',
'dropout',
'split',
'ctc_greedy_decoder',
'edit_distance',
'l2_normalize',
'matmul',
'topk',
'warpctc',
'sequence_reshape',
'transpose',
'im2sequence',
'nce',
'beam_search',
'row_conv',
'multiplex',
'layer_norm',
'softmax_with_cross_entropy',
'smooth_l1',
'one_hot',
'autoincreased_step_counter',
'reshape',
'lod_reset',
'lrn',
'pad',
'label_smooth',
'roi_pool',
'dice_loss',
'image_resize',
'image_resize_short',
'resize_bilinear',
'gather',
'random_crop',
'mean_iou',
'relu',
'log',
] ]
...@@ -3257,8 +3310,7 @@ def topk(input, k, name=None): ...@@ -3257,8 +3310,7 @@ def topk(input, k, name=None):
return values, indices return values, indices
def edit_distance(input, label, normalized=True, ignored_tokens=None, def edit_distance(input, label, normalized=True, ignored_tokens=None):
name=None):
""" """
EditDistance operator computes the edit distances between a batch of EditDistance operator computes the edit distances between a batch of
hypothesis strings and their references. Edit distance, also called hypothesis strings and their references. Edit distance, also called
...@@ -3272,21 +3324,21 @@ def edit_distance(input, label, normalized=True, ignored_tokens=None, ...@@ -3272,21 +3324,21 @@ def edit_distance(input, label, normalized=True, ignored_tokens=None,
"kitten" -> "sitten" -> "sittin" -> "sitting" "kitten" -> "sitten" -> "sittin" -> "sitting"
Input(Hyps) is a LoDTensor consisting of all the hypothesis strings with The input is a LoDTensor consisting of all the hypothesis strings with
the total number denoted by `batch_size`, and the separation is specified the total number denoted by `batch_size`, and the separation is specified
by the LoD information. And the `batch_size` reference strings are arranged by the LoD information. And the `batch_size` reference strings are arranged
in order in the same way in the LoDTensor Input(Refs). in order in the same way in the input LoDTensor.
Output(Out) contains the `batch_size` results and each stands for the edit The output contains the `batch_size` results and each stands for the edit
distance for a pair of strings respectively. If Attr(normalized) is true, distance for a pair of strings respectively. If Attr(normalized) is true,
the edit distance will be divided by the length of reference string. the edit distance will be divided by the length of reference string.
Args: Args:
input(Variable): The indices for hypothesis strings. input(Variable): The indices for hypothesis strings.
label(Variable): The indices for reference strings. label(Variable): The indices for reference strings.
normalized(bool): Indicated whether to normalize the edit distance by normalized(bool, default True): Indicated whether to normalize the edit distance by
the length of reference string. the length of reference string.
ignored_tokens(list of int): Tokens that should be removed before ignored_tokens(list<int>, default None): Tokens that should be removed before
calculating edit distance. calculating edit distance.
name (str): The name of this layer. It is optional. name (str): The name of this layer. It is optional.
...@@ -3298,7 +3350,6 @@ def edit_distance(input, label, normalized=True, ignored_tokens=None, ...@@ -3298,7 +3350,6 @@ def edit_distance(input, label, normalized=True, ignored_tokens=None,
x = fluid.layers.data(name='x', shape=[8], dtype='float32') x = fluid.layers.data(name='x', shape=[8], dtype='float32')
y = fluid.layers.data(name='y', shape=[7], dtype='float32') y = fluid.layers.data(name='y', shape=[7], dtype='float32')
cost = fluid.layers.edit_distance(input=x,label=y) cost = fluid.layers.edit_distance(input=x,label=y)
""" """
helper = LayerHelper("edit_distance", **locals()) helper = LayerHelper("edit_distance", **locals())
...@@ -3418,35 +3469,33 @@ def warpctc(input, label, blank=0, norm_by_times=False): ...@@ -3418,35 +3469,33 @@ def warpctc(input, label, blank=0, norm_by_times=False):
input tensor. input tensor.
Args: Args:
input(Variable): (LodTensor, default: LoDTensor<float>), input (Variable): The unscaled probabilities of variable-length sequences,
the unscaled probabilities of variable-length sequences, which is a 2-D Tensor with LoD information.
which is a 2-D Tensor with LoD information. It's shape is [Lp, num_classes + 1], where Lp is the sum of all input
It's shape is [Lp, num_classes + 1], where Lp is the sum of all input sequences' length and num_classes is the true number of classes.
sequences' length and num_classes is the true number of classes. (not including the blank label).
(not including the blank label). label (Variable): The ground truth of variable-length sequence,
label(Variable): (LodTensor, default: LoDTensor<int>), the ground truth which is a 2-D Tensor with LoD information. It is of the shape [Lg, 1],
of variable-length sequence, which is a 2-D Tensor with LoD where Lg is th sum of all labels' length.
information. It is of the shape [Lg, 1], where Lg is th sum of blank (int, default 0): The blank label index of Connectionist
all labels' length. Temporal Classification (CTC) loss, which is in the
blank (int): default 0, the blank label index of Connectionist half-opened interval [0, num_classes + 1).
Temporal Classification (CTC) loss, which is in the norm_by_times(bool, default false): Whether to normalize the gradients
half-opened interval [0, num_classes + 1). by the number of time-step, which is also the sequence's length.
norm_by_times (bool): default false, whether to normalize There is no need to normalize the gradients if warpctc layer was
the gradients by the number of time-step, which is also the follewed by a mean_op.
sequence's length. There is no need to normalize the gradients
if warpctc layer was follewed by a mean_op.
Returns: Returns:
Variable: The Connectionist Temporal Classification (CTC) loss, Variable: The Connectionist Temporal Classification (CTC) loss,
which is a 2-D Tensor of the shape [batch_size, 1]. which is a 2-D Tensor of the shape [batch_size, 1].
Examples: Examples:
.. code-block:: python .. code-block:: python
y = layers.data(
name='y', shape=[11, 8], dtype='float32', lod_level=1) label = fluid.layers.data(shape=[11, 8], dtype='float32', lod_level=1)
y_predict = layers.data( predict = fluid.layers.data(shape=[11, 1], dtype='float32')
name='y_predict', shape=[11, 1], dtype='float32') cost = fluid.layers.warpctc(input=predict, label=label)
cost = layers.warpctc(input=y_predict, label=y)
""" """
helper = LayerHelper('warpctc', **locals()) helper = LayerHelper('warpctc', **locals())
...@@ -3475,17 +3524,21 @@ def sequence_reshape(input, new_dim): ...@@ -3475,17 +3524,21 @@ def sequence_reshape(input, new_dim):
.. code-block:: text .. code-block:: text
x is a LoDTensor: x is a LoDTensor:
x.lod = [[2, 4]] x.lod = [[0, 2, 6]]
x.data = [[1, 2], [3, 4], x.data = [[1, 2], [3, 4],
[5, 6], [7, 8], [9, 10], [11, 12]] [5, 6], [7, 8],
[9, 10], [11, 12]]
x.dims = [6, 2] x.dims = [6, 2]
set new_dim = 4 set new_dim = 4
then out is a LoDTensor: then out is a LoDTensor:
out.lod = [[1, 2]]
out.data = [[1, 2, 3, 4], out.lod = [[0, 1, 3]]
[5, 6, 7, 8], [9, 10, 11, 12]]
out.data = [[1, 2, 3, 4],
[5, 6, 7, 8],
[9, 10, 11, 12]]
out.dims = [3, 4] out.dims = [3, 4]
Currently, only 1-level LoDTensor is supported and please make sure Currently, only 1-level LoDTensor is supported and please make sure
...@@ -3493,19 +3546,19 @@ def sequence_reshape(input, new_dim): ...@@ -3493,19 +3546,19 @@ def sequence_reshape(input, new_dim):
no remainder for each sequence. no remainder for each sequence.
Args: Args:
input (Variable): (LodTensor, default: LoDTensor<float>), a 2-D LoDTensor
with shape being [N, M] where M for dimension. input (Variable): A 2-D LoDTensor with shape being [N, M] where M for dimension.
new_dim (int): New dimension which the input LoDTensor is reshaped to. new_dim (int): New dimension that the input LoDTensor is reshaped to.
Returns: Returns:
Variable: Reshaped LoDTensor according to new dimension. Variable: Reshaped LoDTensor according to new dimension.
Examples: Examples:
.. code-block:: python .. code-block:: python
x = fluid.layers.data(name='x', shape=[5, 20], x = fluid.layers.data(shape=[5, 20], dtype='float32', lod_level=1)
dtype='float32', lod_level=1) x_reshaped = fluid.layers.sequence_reshape(input=x, new_dim=10)
x_reshaped = layers.sequence_reshape(input=x, new_dim=10)
""" """
helper = LayerHelper('sequence_reshape', **locals()) helper = LayerHelper('sequence_reshape', **locals())
out = helper.create_tmp_variable(helper.input_dtype()) out = helper.create_tmp_variable(helper.input_dtype())
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册