* Remove the error in the elementwise op, use the backup mode to calculate

test=develop

test=develop

* Add InitializeGPUAndLoadModel to solve random hang when downloading sparse parameters.
* Update SaveBase to solve test problem.

* add new macro BOOST_GET_SAFELY & unittests, test=develop

* add different macro type, test=develop

* fix get macro type in executor, test=develop

* four macro part change backup

* using one macro for all case, test=develop

* revert attribute change, test=develop

* change to three func to solve gcc4.8 bug, test=develop

* polish some details, test=develop

CreateStepScopes in recurrent op also clears scopes, which can cause segmentation fault un multi-threading. We add a lock in this PR but it may slow the computation process. We will fix in another way in next PR.

* raise index error when slice out of range; test=develop

* add uni test; test=develop

* fix format error; test=develop

* add comment for py::index_error; test=develop

* polish error message; test=develop

* polish error message; test=develop

* Enabled quantize all and skip missing in QAT

* support to train in static

* support to independent decorator

* remove in_dygraph_mode condition in ProgramTranslator

* fix import param_guard and add train/eval test=develop

* Modify into ShareVarsFromScope and rm __all__ in partial_program test=develop

1. To make ProgramTranslator to support `assert` grammar, this PR adds `assert` python API and C++ code. 

2. Fix a bug: graph_pattern_detector.h #include <gtest/gtest_prod.h> but didn't declared dependency at CMakeLists, which can cause single build failure.

3. Refactoring `Formatter` in print_op to make it reusable and reuse the formatter to print in assert op.

test=develop
test=win_gpu

* add UT for mkldnn_cache_capacity

test=develop

* fix comparison of integer expressions of different signedness

test=develop

* API/OP (margin_rank_loss, nce, row_conv, glu, positive_negative_pair) error message enhancement, test=develop

* add unittest for glu, test=develop

* Replace dygraph_to_static_func with @declarative or program_translator.get_func in test_list.py

* Add comments in ConditionalBlock.

* Support list pop last item. 

* Support pop the i-th item. 

* Support an empty tensor array as Input in assign op and set the kernel type is float.

* solve mklml memory leak on windows

test=develop

* remove unused msvcr120.dll

test=develop

* improve activation ops performance, test=develop

* use 32bit only GPU computation, test=develop

* add timeout and http store in communication, add revert and confirm in fleet
* test=develop

* Fix cusolver loader for Windows in dynamic_loader.cc. test=develop

* Fix missing CUSOLVER_ROUTINE_EACH_R1.
test=gpu
test=develop

* Add unsupprot for cusolver on Windows temporarily. test=develop

* Fix GetCusolverDsoHandle error message. test=develop

Add the pslib graph batch reader mode, add the test case for this change

* Support LoDTensorArray for slice op.
* Support read elements of list in dygraph_to_static
* Fix infershape add test for infershape.
* Support Tensor for Attr(starts) and Attr(ends). 
* Use new interfaces in VarTypeInference. 

fix kron_op: when only one input needs gradient

* enhance add_position_encoding error message, test=develop

* enhance warpctc & scaled_dot_product_attention error message, test=develop

* modified error message and ctest of scaled_dot_product_attention, test=develop

* update the analysis predictor, test=develop

* update the unit test, test=develop

* no priority set before the inferface determined, test=develop

* interface name generalization, test=develop

* update local data preprocess doc

* update for 2.0 QAT

test=develop

test=document_fix

* update benchmark data

test=develop

test=document_fix
Co-authored-by: NWojciech Uss <wojciech.uss@intel.com>

* remove double registery for pyramid_hash op

* add noavx_axpy and noavx_axpy_noadd

test=develop

test=develop

* add print transformer & unify print format, test=develop

* remove using of dygraph_to_static_func, test=develop

* remove python stdout capture, test=develop

* fix compatibility problems for PY2, test=develop

* fix detail error, test=develop

* fix type analysis bug, test=develop

* fix print tuple compatible error in PY2, test=develop

* replace get_func to declarative, test=develop

* fix detail bug, test=develop

* fix some detail problems, test=develop

* change visit_call in print transformer, test=develop

* add trace op, test=develop

* Optimized the implementation of trace op, test=develop

* fix a bug of include in trace_op.h, test=develop

* move trace API from creation to math, test=develop

* modified en doc. test=develop

* add complex trace api

* add complex sum api, test=develop

* modified en doc of complex sum and trace, test=develop

* modified doc and trace API, test=develop

* modified en doc of trace and sum, test=develop

* modified comment in complex kron API, test=develop

* OP Should Not Have Unused Input, test=develop

* add GetExpectedKernelType, test=develop

c++ API ( average_accumulates, tensor_array_to_tensor and average_accumulates) error message enhance. test=develop (#23631)

* add batch fc op, test=develop

* add batch_fc_op, test=develop

* fix untest, test=develop

* rm check_dygraph, test=develop

* fix comment, test=develop

* fix comment, test=develop