* Add default cudnn lib path (#25175)

* add default cudnn lib path, test=develop

* change default path in func, test=develop

* move to linux branch, test=develop

* fix var error in other plat, test=develop

* Refactor dynamic dso search functions (#25214)

* refactor dynamic dso search func, test=develop

* polish details, test=develop

* polish detail based review comments, test=develop

* revert string type change, test=develop

* Polish install error hint message (#25531)

* polish install error hint msg, test=develop

* fix variable error, test=develop

* polish hint messgae again

test=release/2.0
Co-authored-by: NJacek Czaja <jacek.czaja@intel.com>

* cherry-pick of DeviceContext Split, test=develop (#23737)

* New feature: thread local allocator, test=develop (#23989)

* add the thread_local_allocator, test=develop

* refactor the thread_local_allocator, test=develop

* provides option setting strategy, test=develop

* add boost dependency to cuda_stream, test=develop

* declare the stream::Priority as enum class, test=develop

* deal with PADDLE_ENFORCE_CUDA_SUCCESS macro in pr #23816

* cherry-pick,Optimize the error messages of paddle CUDA API

* fix the error messages of paddle CUDA API

* Refactoring PADDLE_ENFORCE_CUDA_SUCCESS, and apply to curand/cudnn/cublas/NCCL

* remove build_ex_string

eagerly release cuda resources before cuda enviroment destroying
test=develop

* Initial FP32 DNNL MatMul Implementation

* Implement int8 DNNL MatMul

* Unify in-kernel-naming, clean UTs

* MatmuL: Introduce op caching

* Final adjustments

test=develop

* Remove dy_graph disablement

test=develop

* Change dnnl header name to new one

test=develop

* Contrain multi head check to prevent fails

test=develop

* Resolve dnnl header problems on MAC CI

* Variable namings to kernel and skip_grad_ci added

test=develop

* Prevent MAC CI from failing

* Prevent windows build from failing

test=develop

* Modify UTs to conform to the rules

* Modify MatMul aux functions namings

test=develop

add addmm op

* Remove the NGraph engine from PDPD repository
1. Each operator was removed from the operator's directory
2. Each test was removed from the unittest directory
3. The parallel executor support was removed from the PDPD
4. The CMake file was removed from the PDPD
5. The NG flags were removed from the repository
test=develop

* Remove ngraph from:
1. Cmake file
2. Python file
test=develop

* delete invalid check inferface Ref & VectorRef, test=develop

* fix vector ref delete error, test=develop

* try the new check inferface, test=develop

* change all related code with new check macro, test=develop

* remove static assert, test=develop

* polish detail, test=develop

* skip coverage problem, test=develop

* add new check macro, test=develop

* fix init_gflags with 'python -c', test=develop

* add test, test=develop

* use sys.executable instead of python, test=develop

* keep dummy, test=develop

* add op inout check macro, test=develop

* fix enforce_test, test=develop

* Delete is_test from activation operators
test=develop

* Revent unneeded changes
test=develop

As nccl comm is not created by CUDADeviceContext, it should be destroyed by the creator as the best practice of RAII.

* refine output of profiler for child event 

initialize global nccl context in dygraph
test=develop

* refine the profiler print test=develop

* add recorded cuda memory apis, fix typo, test=develop

* add more ut, test=develop

* follow comments, test=develop

* fix py35 incompatible issues, test=develop

* add profiler_help.h to refine the code test=develop

* add framework overhead ratio, test=develop

* print GpuMemcpy overhead, test=develop

* add support for the driver api callback and fix the profiler name show bug