* Init complex number neural network, test=release/2.0

* Improve doc writing

* Fix elementwise add & sub

* Fix elementwise mul act

* a) add ut for complex variable; b) remove arg act in elementwise_ops

* cherry-pick,Optimize the error messages of paddle CUDA API

* fix the error messages of paddle CUDA API

* Refactoring PADDLE_ENFORCE_CUDA_SUCCESS, and apply to curand/cudnn/cublas/NCCL

* remove build_ex_string

add manual_seed op into paddle.framework.random

* Fix judge pslib transpiler cherry-pick (#23720)

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* aAdd new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Remove the NGraph engine from PDPD repository
1. Each operator was removed from the operator's directory
2. Each test was removed from the unittest directory
3. The parallel executor support was removed from the PDPD
4. The CMake file was removed from the PDPD
5. The NG flags were removed from the repository
test=develop

* Remove ngraph from:
1. Cmake file
2. Python file
test=develop

* add functional conv

* add test and doc for function convs, test=develop

* update ConvTransposeOp's InferShape and error message, test=develop

* add L1Loss

* support L1Loss, test=develop

* add test, test=develop

* fix batch, test=develop

* follow comments, test=develop

* get pslib version
* test=develop

This reverts commit 66fce9e8.

Fleet Parameter Server API Integrated

This PR provides very basic and simple framework for transforming Dygraph to Static Graph.

API names, final outputs are not determined yet. Feel free to modify or add class/function/type when you think the framework is not extendable for you.

* Fix openblas to support compile on Windows when WITH_MKL=OFF

* MKLDNN v1.0 rebase to Paddle 1.6
test=develop

* Add hacky paddle::string::to_string() implementation

* vectorize<int64-t>() -> vectorize() cleanup
test=develop

* PADDLE_ENFORCE and void_cast fixes
test=develop

* Rebase changes
test=develop

* Cosmetics
test=develop

* Delete MKL from mkldnn.cmake
test=develop

* CMake debug commands
test=develop

* Delete MKLDNN_VERBOSE and rebase fixes
test=develop

* Rebase fixes
test=develop

* Temporarily disable int8 resnet101 vgg16 and vgg19 tests
test=develop

* Add libmkldnn.so.1 to python setup
test=develop

* Add libmkldnn.so.1 to inference_lib cmake after rebase
test=develop

* Post rebase fixes + FC int8 changes
test=develop

* Fix LRN NHWC
test=develop

* Fix NHWC conv3d
test=develop

* Windows build fix + next conv3d fix
test=develop

* Fix conv2d on AVX2 machines
test=develop

* fix compile error in some python version, test=develop

* remove redudant code, test=develop

* fix mkldnn include. test=develop

* fix mkldnn inlcude. test=develop

* Fix TensorRT detection bug

1. Add new search path for TensorRT at tensorrt.cmake
2. Add better debug message
3. Fix the bug of detection of TensorRT version

In NVIDIA official docker image, TensorRT headers are located at
`/usr/include/x86_64-linux-gnu` and TensorRT libraries are located
at `/usr/lib/x86_64-linux-gnu`, so using `-DTENSORRT_ROOT` will
fail to detect TensorRT.

There is no debug/warning message to tell developer that TensorRT
is failed to be detected.

In later version of TensorRT (e.g. v6), `NV_TENSORRT_MAJOR` is
defined at `NvInferVersion.h` instead of `NvInfer.h`, so add
compatibility fix.

* Fix TensorRT variables in CMake

1. Replace `${TENSORRT_ROOT}/include` with `${TENSORRT_INCLUDE_DIR}`
2. Replace `${TENSORRT_ROOT}/lib` with `${TENSORRT_LIBRARY}`

Manually type path may locate incorrect path of TensorRT. Use the
paths detected by system instead.

* Fix TensorRT library path

1. Add new variable - `${TENSORRT_LIBRARY_DIR}`
2. Fix TensorRT library path

inference_lib.cmake and setup.py.in need the path of TensorRT library
instead of the file of TensorRT library, so add new variable to fix it.

* Add more general search rule for TensoRT

Let system detect architecture instead of manually assign it, so
replace `x86_64-linux-gnu` with `${CMAKE_LIBRARY_ARCHITECTURE}`.

* Add more general search rule for TensorRT

Remove duplicate search rules for TensorRT libraries. Use
`${TENSORRT_LIBRARY_DIR}` to get full path of libnvinfer.so

test=develop

* fix windows python setup issue caused by #20641, test=develop

* tweak python/CMakeLists.txt, test=develop

* redirect log inside setup.py, test=develop

* fix typo, test=develop

* Simplify third_party path in Python package
* Fix typo

* add dlpack include in third_party. test=develop

test=develop

* How to write custom op needs to follow framework OP spec.
* Package fluid_framework.so and headers into whl.
* Add paddle.sysconfig.get_include() and paddle.sysconfig.get_lib() to get include dir and lib dir.
* Export some C-APIs to merge OpInfo between core.so and custom_op.so.
* Add unit testing.
* Update API.spec.

* fix libps.so path problem of  1/2/3 dir and third_party
* test = develop

* add tensorrt support for windows

* add fleet util (fleet/utils/fleet_util.py): functions for users' convenience
* add some interface in hdfs util : hdfs is_file、hdfs cat

* fix warpctc.dll not found issue, test=develop

* revert the linux platform change, test=develop

* delete warpctc_lib_path.h.in, test=develop

* add SetPySitePackagePath function

* fix warpctc.dylib not found issue on Mac, test=develop

* improve the paddle lib path setting logic, test=develop

* fix mac ci issue caused by test_warpctc_op unittest, test=develop

* tweak code, test=develop

* fix save/load in Fleet
* add UT framework of Fleet

* support avx and noavx core

* add catch and give some log

test=develop

* fix build

test=develop

* add missing package

test=develop

* fix pybind name

test=develop

* fix import error

test=develop

* conbime noavx core

test=develop

* add requirements

test=develop

* fix unkown message

test=develop

* fix api spec

test=develop

* refine and clean

test=develop

* update

* pass dist ut

* follow comments

test=develop

* refine scripts

test=develop

* Add auto pruning strategy.
1. Fix compressor.
2. Enhence graph executor.
3. Add SAController
4. Add auto pruning strategy.
5. Add unitest for auto pruning strategy.
test=develop

* Init light-nas

* Add light nas.

* Some fix.
test=develop

* Fix sa controller.
test=develop

* Fix unitest of light nas.
test=develop

* Fix setup.py.in and API.spec.
test=develop

* Fix unitest.
1. Fix unitest on windows.
2. Fix package importing in tests directory.

* 1. Remove unused comments.
2. Expose eval_epoch option.
3. Remove unused function in search_agent.
4. Expose max_client_num to yaml file.
5. Move flops constraint to on_epoch_begin function
test=develop

* Fix light nas strategy.
test=develop

* Make controller server stable.
test=develop

* 1. Add try exception to compressor.
2. Remove unitest of light-nas for windows.
test=develop

* Add comments
Enhence controller
test=develop

* Fix comments.
test=develop

* fix the dll not found issue on windows

* Enhance fused_elementwise_activation op.
test=develop

* Move the api fused_elementwise_activation to contrib.
test=develop

* Add including files.
test=develop

* Add the support of sigmoid in fused_elementwise_activetion op.

* Update API.spec.
test=develop

* Init mixed precision training interface

* Add fp16 test script

test=develop

* All initializers support float16

test=develop

* Code cleanup & add more code annotations

test=develop

* Update API spec

test=develop

* Add usage example in doc

test=develop