NLP¶

sequence_conv_pool¶

paddle.trainer_config_helpers.networks.sequence_conv_pool(*args, **kwargs)

Text convolution pooling layers helper.

Text input => Context Projection => FC Layer => Pooling => Output.

Parameters:	name (basestring) – name of output layer(pooling layer name) input (LayerOutput) – name of input layer context_len (int) – context projection length. See context_projection’s document. hidden_size (int) – FC Layer size. context_start (int or None) – context projection length. See context_projection’s context_start. pool_type (BasePoolingType.) – pooling layer type. See pooling_layer’s document. context_proj_layer_name (basestring) – context projection layer name. None if user don’t care. context_proj_param_attr (ParameterAttribute or None.) – context projection parameter attribute. None if user don’t care. fc_layer_name (basestring) – fc layer name. None if user don’t care. fc_param_attr (ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care. fc_bias_attr (ParameterAttribute or None) – fc bias parameter attribute. False if no bias, None if user don’t care. fc_act (BaseActivation) – fc layer activation type. None means tanh pool_bias_attr (ParameterAttribute or None.) – pooling layer bias attr. None if don’t care. False if no bias. fc_attr (ExtraLayerAttribute) – fc layer extra attribute. context_attr (ExtraLayerAttribute) – context projection layer extra attribute. pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
Returns:	output layer name.
Return type:	LayerOutput

text_conv_pool¶

paddle.trainer_config_helpers.networks.text_conv_pool(*args, **kwargs)

Text convolution pooling layers helper.

Text input => Context Projection => FC Layer => Pooling => Output.

Parameters:	name (basestring) – name of output layer(pooling layer name) input (LayerOutput) – name of input layer context_len (int) – context projection length. See context_projection’s document. hidden_size (int) – FC Layer size. context_start (int or None) – context projection length. See context_projection’s context_start. pool_type (BasePoolingType.) – pooling layer type. See pooling_layer’s document. context_proj_layer_name (basestring) – context projection layer name. None if user don’t care. context_proj_param_attr (ParameterAttribute or None.) – context projection parameter attribute. None if user don’t care. fc_layer_name (basestring) – fc layer name. None if user don’t care. fc_param_attr (ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care. fc_bias_attr (ParameterAttribute or None) – fc bias parameter attribute. False if no bias, None if user don’t care. fc_act (BaseActivation) – fc layer activation type. None means tanh pool_bias_attr (ParameterAttribute or None.) – pooling layer bias attr. None if don’t care. False if no bias. fc_attr (ExtraLayerAttribute) – fc layer extra attribute. context_attr (ExtraLayerAttribute) – context projection layer extra attribute. pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
Returns:	output layer name.
Return type:	LayerOutput

Images¶

img_conv_bn_pool¶

paddle.trainer_config_helpers.networks.img_conv_bn_pool(*args, **kwargs)

Convolution, batch normalization, pooling group.

Parameters:	name (basestring) – group name input (LayerOutput) – layer’s input filter_size (int) – see img_conv_layer’s document num_filters (int) – see img_conv_layer’s document pool_size (int) – see img_pool_layer’s document. pool_type (BasePoolingType) – see img_pool_layer’s document. act (BaseActivation) – see batch_norm_layer’s document. groups (int) – see img_conv_layer’s document conv_stride (int) – see img_conv_layer’s document. conv_padding (int) – see img_conv_layer’s document. conv_bias_attr (ParameterAttribute) – see img_conv_layer’s document. num_channel (int) – see img_conv_layer’s document. conv_param_attr (ParameterAttribute) – see img_conv_layer’s document. shared_bias (bool) – see img_conv_layer’s document. conv_layer_attr (ExtraLayerOutput) – see img_conv_layer’s document. bn_param_attr (ParameterAttribute.) – see batch_norm_layer’s document. bn_bias_attr – see batch_norm_layer’s document. bn_layer_attr – ParameterAttribute. pool_stride (int) – see img_pool_layer’s document. pool_start (int) – see img_pool_layer’s document. pool_padding (int) – see img_pool_layer’s document. pool_layer_attr (ExtraLayerAttribute) – see img_pool_layer’s document.
Returns:	Layer groups output
Return type:	LayerOutput

img_conv_group¶

paddle.trainer_config_helpers.networks.img_conv_group(*args, **kwargs)

Image Convolution Group, Used for vgg net.

TODO(yuyang18): Complete docs

Parameters:	conv_batchnorm_drop_rate – input – conv_num_filter – pool_size – num_channels – conv_padding – conv_filter_size – conv_act – conv_with_batchnorm – pool_stride – pool_type –
Returns:

simple_img_conv_pool¶

paddle.trainer_config_helpers.networks.simple_img_conv_pool(*args, **kwargs)

Simple image convolution and pooling group.

Input => conv => pooling

Parameters:	name (basestring) – group name input (LayerOutput) – input layer name. filter_size (int) – see img_conv_layer for details num_filters (int) – see img_conv_layer for details pool_size (int) – see img_pool_layer for details pool_type (BasePoolingType) – see img_pool_layer for details act (BaseActivation) – see img_conv_layer for details groups (int) – see img_conv_layer for details conv_stride (int) – see img_conv_layer for details conv_padding (int) – see img_conv_layer for details bias_attr (ParameterAttribute) – see img_conv_layer for details num_channel (int) – see img_conv_layer for details param_attr (ParameterAttribute) – see img_conv_layer for details shared_bias (bool) – see img_conv_layer for details conv_layer_attr (ExtraLayerAttribute) – see img_conv_layer for details pool_stride (int) – see img_conv_layer for details pool_start (int) – see img_conv_layer for details pool_padding (int) – see img_conv_layer for details pool_layer_attr (ExtraLayerAttribute) – see img_conv_layer for details
Returns:	Layer’s output
Return type:	LayerOutput

vgg_16_network¶

paddle.trainer_config_helpers.networks.vgg_16_network(input_image, num_channels, num_classes=1000)

Same model from https://gist.github.com/ksimonyan/211839e770f7b538e2d8

Parameters:	num_classes – input_image (LayerOutput) – num_channels (int) –
Returns:

Recurrent¶

LSTM¶

lstmemory_unit¶

paddle.trainer_config_helpers.networks.lstmemory_unit(*args, **kwargs)

TODO(yuyang18): complete docs

Parameters:	input (LayerOutput) – input layer name. name (basestring) – lstmemory unit name. size (int) – lstmemory unit size. param_attr (ParameterAttribute) – Parameter config, None if use default. act (BaseActivation) – lstm final activate type gate_act (BaseActivation) – lstm gate activate type state_act (BaseActivation) – lstm state activate type. mixed_bias_attr (ParameterAttribute\|False) – bias parameter attribute of mixed layer. False means no bias, None means default bias. lstm_bias_attr (ParameterAttribute\|False) – bias parameter attribute of lstm layer. False means no bias, None means default bias. mixed_layer_attr (ExtraLayerAttribute) – mixed layer’s extra attribute. lstm_layer_attr (ExtraLayerAttribute) – lstm layer’s extra attribute. get_output_layer_attr (ExtraLayerAttribute) – get output layer’s extra attribute.
Returns:	lstmemory unit name.
Return type:	LayerOutput

lstmemory_group¶

paddle.trainer_config_helpers.networks.lstmemory_group(*args, **kwargs)

TODO(yuyang18): complete docs

Parameters:	input (LayerOutput) – input layer name. name (basestring) – lstmemory group name. size (int) – lstmemory group size. reverse (bool) – is lstm reversed param_attr (ParameterAttribute) – Parameter config, None if use default. act (BaseActivation) – lstm final activate type gate_act (BaseActivation) – lstm gate activate type state_act (BaseActivation) – lstm state activate type. mixed_bias_attr (ParameterAttribute\|False) – bias parameter attribute of mixed layer. False means no bias, None means default bias. lstm_bias_attr (ParameterAttribute\|False) – bias parameter attribute of lstm layer. False means no bias, None means default bias. mixed_layer_attr (ExtraLayerAttribute) – mixed layer’s extra attribute. lstm_layer_attr (ExtraLayerAttribute) – lstm layer’s extra attribute. get_output_layer_attr (ExtraLayerAttribute) – get output layer’s extra attribute.
Returns:	lstmemory group name.
Return type:	LayerOutput

simple_lstm¶

paddle.trainer_config_helpers.networks.simple_lstm(*args, **kwargs)

Simple LSTM Cell.

It just combine a mixed layer with fully_matrix_projection and a lstmemory layer. The simple lstm cell was implemented as follow equations.

\[\begin{split}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\end{split}\]\[\begin{split}f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\end{split}\]\[\begin{split}c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\end{split}\]\[\begin{split}o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\end{split}\]\[\begin{split}h_t & = o_t tanh(c_t)\end{split}\]

Please refer Generating Sequences With Recurrent Neural Networks if you want to know what lstm is. Link is here.

Parameters:	name (basestring) – lstm layer name. input (LayerOutput) – input layer name. size (int) – lstm layer size. reverse (bool) – is lstm reversed mat_param_attr (ParameterAttribute) – mixed layer’s matrix projection parameter attribute. bias_param_attr (ParameterAttribute\|False) – bias parameter attribute. False means no bias, None means default bias. inner_param_attr (ParameterAttribute) – lstm cell parameter attribute. act (BaseActivation) – lstm final activate type gate_act (BaseActivation) – lstm gate activate type state_act (BaseActivation) – lstm state activate type. mixed_layer_attr (ExtraLayerAttribute) – mixed layer’s extra attribute. lstm_cell_attr (ExtraLayerAttribute) – lstm layer’s extra attribute.
Returns:	lstm layer name.
Return type:	LayerOutput

bidirectional_lstm¶

paddle.trainer_config_helpers.networks.bidirectional_lstm(*args, **kwargs)

TODO(yuyang18): Complete docs

Parameters:	name (basestring) – bidirectional lstm layer name. input (LayerOutput) – input layer. size (int) – lstm layer size. return_seq (bool) – If False, concat word in last time step and return. If True, concat sequnce in all time step and return.
Returns:	lstm layer name.
Return type:	LayerOutput

GRU¶

gru_unit¶

paddle.trainer_config_helpers.networks.gru_unit(*args, **kwargs)

Parameters:	input (LayerOutput) – name – size – gru_bias_attr – act – gate_act – gru_layer_attr –
Returns:

gru_group¶

simple_gru¶

simple_attention¶

paddle.trainer_config_helpers.networks.simple_attention(*args, **kwargs)

Calculate and then return a context vector by attention machanism. Size of the context vector equals to size of the encoded_sequence.

\[\begin{split}a(s_{i-1},h_{j}) & = v_{a}f(W_{a}s_{t-1} + U_{a}h_{j})\end{split}\]\[\begin{split}e_{i,j} & = a(s_{i-1}, h_{j})\end{split}\]\[\begin{split}a_{i,j} & = \frac{exp(e_{i,j})}{\sum_{k=1}^{T_x}{exp(e_{i,k})}}\end{split}\]\[\begin{split}c_{i} & = \sum_{j=1}^{T_{x}}a_{i,j}h_{j}\end{split}\]

where \(h_{j}\) is the jth element of encoded_sequence, \(U_{a}h_{j}\) is the jth element of encoded_proj \(s_{i-1}\) is decoder_state \(f\) is weight_act, and is set to tanh by default.

Please refer to Neural Machine Translation by Jointly Learning to Align and Translate for more details. The link is as follows: https://arxiv.org/abs/1409.0473.

The example usage is:

context = simple_attention(encoded_sequence=enc_seq,
                           encoded_proj=enc_proj,
                           decoder_state=decoder_prev,)

Parameters:

name (basestring) – name of the attention model.
softmax_param_attr (ParameterAttribute) – parameter attribute of sequence softmax that is used to produce attention weight
weight_act (Activation) – activation of the attention model
encoded_sequence (LayerOutput) – output of the encoder
encoded_proj (LayerOutput) – attention weight is computed by a feed forward neural network which has two inputs : decoder’s hidden state of previous time step and encoder’s output. encoded_proj is output of the feed-forward network for encoder’s output. Here we pre-compute it outside simple_attention for speed consideration.
decoder_state (LayerOutput) – hidden state of decoder in previous time step
transform_param_attr (ParameterAttribute) – parameter attribute of the feed-forward network that takes decoder_state as inputs to compute attention weight.

Returns:

a context vector

Miscs¶

dropout_layer¶

paddle.trainer_config_helpers.networks.dropout_layer(*args, **kwargs)

@TODO(yuyang18): Add comments.

Parameters:	name – input – dropout_rate –
Returns:

outputs¶

paddle.trainer_config_helpers.networks.outputs(layers)

Declare the end of network. Currently it will only calculate the input/output order of network. It will calculate the predict network or train network’s output automatically.

Parameters:	layers (list\|tuple\|LayerOutput) –
Returns: