Layers¶

fc¶

paddle.v2.fluid.layers.fc(input, size, num_flatten_dims=1, param_attr=None, bias_attr=None, act=None, name=None)

Fully Connected Layer.

Parameters:

input – The input tensor to the function
size – The size of the layer
num_flatten_dims – Number of columns in input
param_attr – The parameters/weights to the FC Layer
param_initializer – Initializer used for the weight/parameter. If None, XavierInitializer() is used
bias_attr – The bias parameter for the FC layer
bias_initializer – Initializer used for the bias. If None, then ConstantInitializer() is used
act – Activation to be applied to the output of FC layer
name – Name/alias of the function
main_program – Name of the main program that calls this
startup_program – Name of the startup program

This function can take in multiple inputs and performs the Fully Connected function (linear transformation) on top of each of them. So for input x, the output will be : Wx + b. Where W is the parameter, b the bias and x is the input.

The function also applies an activation (non-linearity) on top of the output, if activation is passed in the input.

All the input variables of this function are passed in as local variables to the LayerHelper constructor.

embedding¶

paddle.v2.fluid.layers.embedding(input, size, is_sparse=False, param_attr=None, dtype='float32')

Embedding Layer.

Parameters:	param_initializer – input – The input to the function size – The size of the layer is_sparse – A flag that decleares whether the input is sparse param_attr – Parameters for this layer dtype – The type of data : float32, float_16, int etc main_program – Name of the main program that calls this startup_program – Name of the startup program

This function can take in the input (which is a vector of IDs) and performs a lookup in the lookup_table using these IDs, to result into the embedding of each ID in the input.

All the input variables of this function are passed in as local variables to the LayerHelper constructor.

dynamic_lstm¶

paddle.v2.fluid.layers.dynamic_lstm(input, size, param_attr=None, bias_attr=None, use_peepholes=True, is_reverse=False, gate_activation='sigmoid', cell_activation='tanh', candidate_activation='tanh', dtype='float32')

data¶

paddle.v2.fluid.layers.data(name, shape, append_batch_size=True, dtype='float32', lod_level=0, type=VarType.LOD_TENSOR, stop_gradient=True)

Data Layer.

Parameters:

name – The name/alias of the function
shape – Tuple declaring the shape.
append_batch_size – Whether or not to append the data as a batch.
dtype – The type of data : float32, float_16, int etc
type – The output type. By default it is LOD_TENSOR.
lod_level (int) – The LoD Level. 0 means the input data is not a sequence.
main_program – Name of the main program that calls this
startup_program – Name of the startup program
stop_gradient – A boolean that mentions whether gradient should flow.

This function takes in input and based on whether data has to be returned back as a minibatch, it creates the global variable using the helper functions. The global variables can be accessed by all the following operations and layers in the graph.

All the input variables of this function are passed in as local variables to the LayerHelper constructor.

mean¶

paddle.v2.fluid.layers.mean(**kwargs)

Mean Operator.

Out is a scalar which is the mean of all elements in X.

Parameters:	x – The input of mean op Duplicable: False Optional: False
Returns:	The output of mean op

mul¶

paddle.v2.fluid.layers.mul(**kwargs)

Mul Operator.

This operator is used to perform matrix multiplication for input X and Y.

The equation is:

$$Out = X * Y$$

Both the input X and Y can carry the LoD (Level of Details) information, or not. But the output only shares the LoD information with input X.

Parameters:

x – The first input of mul op Duplicable: False Optional: False
y – The second input of mul op Duplicable: False Optional: False
x_num_col_dims (INT) – (int, default 1) mul_op can take tensors with more than two dimensions as input X, in that case, tensors will be reshaped to a matrix. The matrix’s first dimension(column length) will be the product of tensor’s last num_col_dims dimensions, and the matrix’s second dimension(row length) will be the product of tensor’s first rank - num_col_dims dimensions.
y_num_col_dims (INT) – (int, default 1) mul_op can take tensors with more than two dimensions as input Y, in that case, tensors will be reshaped to a matrix. Just like input X.

Returns:

The output of mul op

elementwise_add¶

paddle.v2.fluid.layers.elementwise_add(**kwargs)

Limited Elementwise Add Operator.

The equation is:

$Out = X + Y$

X is a tensor of any dimension and the dimensions of tensor Y must be smaller than or equal to the dimensions of X.

There are two cases for this operator: 1. The shape of Y is same with X; 2. The shape of Y is a subset of X.

For case 2: Y will be broadcasted to match the shape of X and axis should be the starting dimension index for broadcasting Y onto X.

example

shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5) shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0

Both the input X and Y can carry the LoD (Level of Details) information, or not. But the output only shares the LoD information with input X.

Parameters:	x – (Tensor) The first input tensor of elementwise op Duplicable: False Optional: False y – (Tensor) The second input tensor of elementwise op Duplicable: False Optional: False axis (INT) – (int, default -1) The starting dimension index for broadcasting Y onto X
Returns:	The output of elementwise op

elementwise_div¶

paddle.v2.fluid.layers.elementwise_div(**kwargs)

Limited Elementwise Div Operator.

The equation is:

$Out = X / Y$

X is a tensor of any dimension and the dimensions of tensor Y must be smaller than or equal to the dimensions of X.

There are two cases for this operator: 1. The shape of Y is same with X; 2. The shape of Y is a subset of X.

For case 2: Y will be broadcasted to match the shape of X and axis should be the starting dimension index for broadcasting Y onto X.

example

shape(X) = (2, 3, 4, 5), shape(Y) = (,) shape(X) = (2, 3, 4, 5), shape(Y) = (5,) shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5) shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1 shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0

Both the input X and Y can carry the LoD (Level of Details) information, or not. But the output only shares the LoD information with input X.

Parameters:	x – (Tensor) The first input tensor of elementwise op Duplicable: False Optional: False y – (Tensor) The second input tensor of elementwise op Duplicable: False Optional: False axis (INT) – (int, default -1) The starting dimension index for broadcasting Y onto X
Returns:	The output of elementwise op

dropout¶

paddle.v2.fluid.layers.dropout(**kwargs)

Dropout Operator.

Dropout refers to randomly dropping out units in a nerual network. It is a regularization technique for reducing overfitting by preventing neuron co-adaption during training. The dropout operator randomly set (according to the given dropout probability) the outputs of some units to zero, while others are set equal to their corresponding inputs.

Parameters:	x – The input of dropout op. Duplicable: False Optional: False dropout_prob (FLOAT) – Probability of setting units to zero. is_test (BOOLEAN) – True if in test phase. seed (INT) – Dropout random seed.
Returns:	The output of dropout op.

reshape¶

paddle.v2.fluid.layers.reshape(**kwargs)

Reshape Operator.

Reshape Input(X) into the shape specified by Attr(shape).

An example: Given a 2-D tensor X with 2 rows and 2 columns

[[1, 2], [3, 4]]

and target shape = [1, 4], the reshape operator will transform the tensor X into a 2-D tensor:

[[1, 2, 3, 4]]

One dimension in the target shape can be set -1, representing that its size is unknown. In this case, the real dimension will be infered from the original shape of Input(X) and other dimensions in the target shape.

Parameters:	x – The input tensor of reshape operator. Duplicable: False Optional: False shape (INTS) – (vector<int>) Target shape of reshape operator.
Returns:	The output tensor of reshape operator.

sigmoid¶

paddle.v2.fluid.layers.sigmoid(**kwargs)

Sigmoid Activation Operator

$$y = frac{1}{1 + e^{-x}}$$

Parameters:	x – Input of Sigmoid operator Duplicable: False Optional: False
Returns:	Output of Sigmoid operator

scale¶

paddle.v2.fluid.layers.scale(**kwargs)

Scale operator

$$Out = scale*X$$

Parameters:	x – (Tensor) Input tensor of scale operator. Duplicable: False Optional: False scale (FLOAT) – (float, default 0)The scaling factor of the scale operator.
Returns:	(Tensor) Output tensor of scale operator.

reshape¶

paddle.v2.fluid.layers.reshape(**kwargs)

Reshape Operator.

Reshape Input(X) into the shape specified by Attr(shape).

An example: Given a 2-D tensor X with 2 rows and 2 columns

[[1, 2], [3, 4]]

and target shape = [1, 4], the reshape operator will transform the tensor X into a 2-D tensor:

[[1, 2, 3, 4]]

One dimension in the target shape can be set -1, representing that its size is unknown. In this case, the real dimension will be infered from the original shape of Input(X) and other dimensions in the target shape.

Parameters:	x – The input tensor of reshape operator. Duplicable: False Optional: False shape (INTS) – (vector<int>) Target shape of reshape operator.
Returns:	The output tensor of reshape operator.

transpose¶

paddle.v2.fluid.layers.transpose(**kwargs)

Transpose Operator.

The input tensor will be permuted according to the axis values given. The op functions similar to how numpy.transpose works in python. For example:

>> input = numpy.arange(6).reshape((2,3)) >> input array([[0, 1, 2],

[3, 4, 5]])

>> axis = [1, 0] >> output = input.transpose(axis) >> output array([[0, 3],

[1, 4],

[2, 5]])

So, given a input tensor of shape(N, C, H, W) and the axis is {0, 2, 3, 1}, the output tensor shape will be (N, H, W, C)

Parameters:	x – (Tensor)The input tensor, tensors with rank at most 6 are supported Duplicable: False Optional: False axis (INTS) – (vector<int>)A list of values, and the size of the list should be the same with the input tensor rank, the tensor will permute the axes according the the values given
Returns:	(Tensor)The output tensor

sigmoid_cross_entropy_with_logits¶

cast¶

paddle.v2.fluid.layers.cast(x, dtype): This function takes in the input with input_dtype and casts it to the output_dtype as the output.

concat¶

paddle.v2.fluid.layers.concat(input, axis): This function concats the input along the axis mentioned and returns that as the output.

sums¶

paddle.v2.fluid.layers.sums(input, out=None): This function takes in the input and performs the sum operation on it and returns that as the output.

linear_chain_crf¶

paddle.v2.fluid.layers.linear_chain_crf(input, label, param_attr=None)

assign¶

paddle.v2.fluid.layers.embedding(input, size, is_sparse=False, param_attr=None, dtype='float32')

Embedding Layer.

Parameters:	param_initializer – input – The input to the function size – The size of the layer is_sparse – A flag that decleares whether the input is sparse param_attr – Parameters for this layer dtype – The type of data : float32, float_16, int etc main_program – Name of the main program that calls this startup_program – Name of the startup program

This function can take in the input (which is a vector of IDs) and performs a lookup in the lookup_table using these IDs, to result into the embedding of each ID in the input.

All the input variables of this function are passed in as local variables to the LayerHelper constructor.

split_lod_tensor¶

paddle.v2.fluid.layers.split_lod_tensor(input, mask, level=0)

merge_lod_tensor¶

paddle.v2.fluid.layers.merge_lod_tensor(in_true, in_false, x, mask, level=0)

cos_sim¶

paddle.v2.fluid.layers.cos_sim(X, Y, **kwargs): This function performs the cosine similarity between two tensors X and Y and returns that as the output.

cross_entropy¶

paddle.v2.fluid.layers.cross_entropy(input, label, **kwargs): This function computes cross_entropy using the input and label.

square_error_cost¶

paddle.v2.fluid.layers.square_error_cost(input, label, **kwargs): This functions returns the squared error cost using the input and label. The output is appending the op to do the above.

accuracy¶

paddle.v2.fluid.layers.accuracy(input, label, k=1, correct=None, total=None, **kwargs): This function computes the accuracy using the input and label. The output is the top_k inputs and their indices.

sequence_conv¶

paddle.v2.fluid.layers.sequence_conv(input, num_filters, filter_size=3, filter_stride=1, padding=None, bias_attr=None, param_attr=None, act=None): This function creates the op for sequence_conv, using the inputs and other convolutional configurations for the filters and stride as given in the input parameters to the function.

conv2d¶

paddle.v2.fluid.layers.conv2d(input, num_filters, filter_size, stride=None, padding=None, groups=None, param_attr=None, bias_attr=None, act=None, name=None): This function creates the op for a 2-dimensional Convolution. This is performed using the parameters of filters(size, dimensionality etc) , stride and other configurations for a Convolution operation. This funciton can also append an activation on top of the conv-2d output, if mentioned in the input parameters.

sequence_pool¶

paddle.v2.fluid.layers.sequence_pool(input, pool_type, **kwargs): This function add the operator for sequence pooling. This is applied on top of the input using pool_type mentioned in the parameters.

pool2d¶

paddle.v2.fluid.layers.pool2d(input, pool_size, pool_type, pool_stride=None, pool_padding=None, global_pooling=False): This function adds the operator for pooling in 2 dimensions, using the pooling configurations mentioned in input parameters.

batch_norm¶

paddle.v2.fluid.layers.batch_norm(input, act=None, is_test=False, momentum=0.9, epsilon=1e-05, param_attr=None, bias_attr=None, data_layout='NCHW'): This function helps create an operator to implement the BatchNorm layer using the configurations from the input parameters.

beam_search_decode¶

paddle.v2.fluid.layers.beam_search_decode(ids, scores)

lod_rank_table¶

paddle.v2.fluid.layers.lod_rank_table(x, level=0): This function creates an operator for creating a LOD_RANK_TABLE using the input x.

max_sequence_len¶

paddle.v2.fluid.layers.max_sequence_len(rank_table): This function creates an operator to calculate the length of max seqence through input rank_table(should be a lod_rank_table)

topk¶

paddle.v2.fluid.layers.topk(input, k)

lod_tensor_to_array¶

paddle.v2.fluid.layers.lod_tensor_to_array(x, table): This function creates an operator to convert an LOD_Tensor to an array.

array_to_lod_tensor¶

paddle.v2.fluid.layers.array_to_lod_tensor(x, table): This function creates an operator to convert an array to a LOD_Tensor.

fill_constant¶

paddle.v2.fluid.layers.fill_constant(shape, dtype, value, out=None): This function creates a tensor , with shape as mentioned in the input and specified dtype and fills this up with a constant value that comes in the input. It also sets the stop_gradient to be True.

fill_constant_batch_size_like¶

paddle.v2.fluid.layers.fill_constant_batch_size_like(input, shape, dtype, value, input_dim_idx=0, output_dim_idx=0)

ones¶

paddle.v2.fluid.layers.ones(shape, dtype): This function performs the same function as fill_constant() declared above with the constant value being 1.0.

zeros¶

paddle.v2.fluid.layers.zeros(shape, dtype): This function performs the same function as fill_constant() declared above with the constant value being 0.0.

increment¶

paddle.v2.fluid.layers.increment(x, value=1.0, in_place=True): This function creates an operator to increment each value in the input x by an amount: value as mentioned in the input parameter. This operation is performed in-place by default.

array_write¶

paddle.v2.fluid.layers.array_write(x, i, array=None): This function creates an operator to write the data out as a LOD_TENSOR_ARRAY.

create_array¶

paddle.v2.fluid.layers.create_array(dtype)

less_than¶

paddle.v2.fluid.layers.less_than(x, y, cond=None, **ignored)

array_read¶

paddle.v2.fluid.layers.array_read(array, i): This function creates an operator to read the data in as a LOD_TENSOR_ARRAY.

shrink_memory¶

paddle.v2.fluid.layers.shrink_memory(x, i, table): This function creates an operator to shrink_rnn_memory using the RankTable as mentioned in the input parameter.

array_length¶

paddle.v2.fluid.layers.array_length(array): This function creates an operator to find the length of the LOD_TENSOR_ARRAY.

conv2d_transpose¶

paddle.v2.fluid.layers.conv2d_transpose(input, num_filters, output_size=None, filter_size=None, padding=None, stride=None, param_attr=None)

The transpose of conv2d layer.

This layer is also known as deconvolution layer.

Parameters:	input (Variable) – The input image with [N, C, H, W] format. num_filters (int) – The number of filter. It is as same as the output image channel. output_size (int\|tuple\|None) – The output image size. If output size is a tuple, it must contain two integers, (image_H, image_W). This parameter only works when filter_size is None. filter_size (int\|tuple\|None) – The filter size. If filter_size is a tuple, it must contain two integers, (filter_size_H, filter_size_W). Otherwise, the filter will be a square. None if use output size to calculate filter_size padding (int\|tuple) – The padding size. If padding is a tuple, it must contain two integers, (padding_H, padding_W). Otherwise, the padding_H = padding_W = padding. stride (int\|tuple) – The stride size. If stride is a tuple, it must contain two integers, (stride_H, stride_W). Otherwise, the stride_H = stride_W = stride. param_attr – Parameter Attribute. main_program (Program) – the main program startup_program (Program) – the startup program
Returns:	Output image.
Return type:	Variable

sequence_expand¶

paddle.v2.fluid.layers.sequence_expand(x, y)

Sequence Expand Layer. This layer will expand the input variable x according to LoD information of y. And the following examples will explain how sequence_expand works:

* Case 1
    x is a LoDTensor:
        x.lod = [[0,       2, 3],
                 [0, 1,    3, 4]]
        x.data = [a, b, c, d]
        x.dims = [4, 1]

    y is a LoDTensor:
        y.lod = [[0,    2,    4],
                 [0, 3, 6, 7, 8]]

    with condition len(y.lod[-1]) - 1 == x.dims[0]

    then output is a 2-level LoDTensor:
        out.lod = [[0,                2,    4],
                   [0,       3,       6, 7, 8]]
        out.data = [a, a, a, b, b, b, c, d]
        out.dims = [8, 1]

* Case 2
    x is a Tensor:
        x.data = [a, b, c]
        x.dims = [3, 1]

    y is a LoDTensor:
        y.lod = [[0, 2, 3, 6]]

    with condition len(y.lod[-1]) - 1 == x.dims[0]

    then output is a 1-level LoDTensor:
        out.lod = [[0,    2, 3,      6]]
        out.data = [a, a, b, c, c, c]
        out.dims = [6, 1]

Parameters:	x (Variable) – The input variable which is a Tensor or LoDTensor. y (Variable) – The input variable which is a LoDTensor.
Returns:	The expanded variable which is a LoDTensor.
Return type:	Variable

Examples

x = fluid.layers.data(name='x', shape=[10], dtype='float32')
y = fluid.layers.data(name='y', shape=[10, 20],
                 dtype='float32', lod_level=1)
out = layers.sequence_expand(x=x, y=y)

lstm_unit¶

paddle.v2.fluid.layers.lstm_unit(x_t, hidden_t_prev, cell_t_prev, forget_bias=0.0, param_attr=None, bias_attr=None)

Lstm unit layer. The equation of a lstm step is:

\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{x_i}x_{t} + W_{h_i}h_{t-1} + W_{c_i}c_{t-1} + b_i)\\f_t & = \sigma(W_{x_f}x_{t} + W_{h_f}h_{t-1} + W_{c_f}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{x_c}x_t+W_{h_c}h_{t-1} + b_c)\\o_t & = \sigma(W_{x_o}x_{t} + W_{h_o}h_{t-1} + W_{c_o}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]

The inputs of lstm unit includes $x_t$, $h_{t-1}$ and $c_{t-1}$. The implementation separates the linear transformation and non-linear transformation apart. Here, we take $i_t$ as an example. The linear transformation is applied by calling a fc layer and the equation is:

\[L_{i_t} = W_{x_i}x_{t} + W_{h_i}h_{t-1} + W_{c_i}c_{t-1} + b_i\]

The non-linear transformation is applied by calling lstm_unit_op and the equation is:

\[i_t = \sigma(L_{i_t})\]

This layer has two outputs including $h_t$ and $o_t$.

Parameters:	x_t (Variable) – The input value of current step. hidden_t_prev (Variable) – The hidden value of lstm unit. cell_t_prev (Variable) – The cell value of lstm unit. forget_bias (float) – The forget bias of lstm unit. param_attr (ParamAttr) – The attributes of parameter weights, used to set initializer, name etc. bias_attr (ParamAttr) – The attributes of bias weights, if not False, bias weights will be created and be set to default value.
Returns:	The hidden value and cell value of lstm unit.
Return type:	tuple
Raises:	`ValueError` – The ranks of x_t, hidden_t_prev and cell_t_prev not be 2 or the 1st dimensions of x_t, hidden_t_prev and cell_t_prev not be the same.

Examples

x_t = fluid.layers.fc(input=x_t_data, size=10)
prev_hidden = fluid.layers.fc(input=prev_hidden_data, size=20)
prev_cell = fluid.layers.fc(input=prev_cell_data, size=30)
hidden_value, cell_value = fluid.layers.lstm_unit(x_t=x_t,
                                       hidden_t_prev=prev_hidden,
                                       cell_t_prev=prev_cell)

sequence_softmax¶

paddle.v2.fluid.layers.sequence_softmax(**kwargs)

Sequence Softmax Operator.

SequenceSoftmaxOp computes the softmax activation among all time-steps for each sequence. The dimension of each time-step should be 1. Thus, the shape of input Tensor can be either [N, 1] or [N], where N is the sum of the length of all sequences.

The algorithm works as follows:

for i-th sequence in a mini-batch:

$$Out(X[lod[i]:lod[i+1]], :) =: frac{exp(X[lod[i]:lod[i+1], :])} {sum(exp(X[lod[i]:lod[i+1], :]))}$$

For example, for a mini-batch of 3 sequences with variable-length, each containing 2, 3, 2 time-steps, the lod of which is [0, 2, 5, 7], then softmax will be computed among X[0:2, :], X[2:5, :], X[5:7, :] and N turns out to be 7.

Parameters:	x – (LoDTensor) 1-D or 2-D input LoDTensor with the 2-nd dimension of length 1. Duplicable: False Optional: False
Returns:	(LoDTensor) 1-D or 2-D output LoDTensor with the 2-nd dimension of length 1.