* added external reorder to profiler

* resolved conflict

* added enable_static

* initial version of lstm, not working yet

* added lstm to operators.cmake

* added vanilla lstm mkldnn op

* added peephole weights integration

* minor changes

* added formatting

* added fusion_lstm_mkldnn to static_whitelist

* added formatting

* removed comment

* moved use_peepholes attribute inside is_cached block

* reverted wrong changes

* minor formatting change

* minor changes

* polish printing dtype

* fix special case

* add abs test case

* use std::abs to fix abs bug

* fix the abs bug

* fix abs bug

* Fix the bug in fleet amp_init.

* Fix the amp_init unit test.

* support reduce_max op on kunlun

* support reduce_max op on kunlun

* support reduce_max op on kunlun

* support reduce_max op on kunlun

* delay the 'timeout' of 'test_static_save_load'.

* delay the 'timeout' of 'test_static_save_load'.

* rewrite abs op

* rewrite abs op and remove abs in activation

* remove abs register in old codes

* fix abs_grad type error

* fix abs double_grad output name error

* modify abs_grad, abs_grad_grad functor for windows building

* format code style

* fix the bug of result is nan when the divisor is zero

* add missing abs attr and add abs for float16

* add fleet amp.init()

* add unittest for fleet_amp_init

* update, test=develop

* add paddle.

* add unittest

* add rmsprop_op_xpu test=kunlun

* modified rmsprop_op_xpu error code. test=kunlun

* support layer_norm fp16 in dygraph amp

* add ut

* refine code

* add 2.0 API: device_guard

Fix potential port conflicts

* Avoid  bug on 'MAC python3.5/6'.

* Choose the saving method according to the OS.

* smaller length of '_unpack_saved_dict' for MAC OS.

* add version information of Python.

* Edit comment.

* Modify the calculation logic of LambOptimizer

* add view strategy on squeeze,unsqueeze,reshape,flatten

* add squeeze unittest

* add unittests

* use View strategy as name rather than Reuse Allacation

* fix view api doc

* fix format

* use core.ops when input of reshape2 is Tensor

* fix test_cross_entropy_loss error because of reshape2

* fix test_cross_entropy_loss error because of reshape2

* add inplace strategy

* add elementwise_add sub

* let backward op not use inplace

* grad op do not use inplace

* fix memory increase error and add leaf error message

* delete selected_rows

* change op_function

* little change

* solve HandleViewBetweenInputAndOutput

* add unittest and leaf error message

* merge view error

* optimize op_function_generator format and support sum inplace op

* fix format of basic_engine

* fix format for framework

* little change of variable wrapper

* add reshape, squeeze, unsqueeze, scatter api

* add relu elu tanh softmax inplace api

* fix test_squeeze_op unittest

* fix test_relu_op unittest

* fix comment problems

* delete sample code of inplace api

* add reference of grad_pending_nodes in basic_engine

* fix unittest name

* add inplace apis into wlist

* fix error message

* add PADDLE_ENFORCE for set grad op twice

* fix head file error

* test=develop, fix fleet.metrics(mse, rmse, mae)

* test=develop, add distributed_infer

As the title

As the title

* optimize memcpy perf for kunlun

* remove useless unitest for kunlun mean

* minor