* add new api (set_global_initializer/reset_global_initializer),test=develop

* add new api (set_global_initializer/reset_global_initializer),test=develop

* fix doc and example code of set_global_initializer,test=develop

* add for iter var list, test=develop

* add enumerate unittest, test=develop

* Start implementing int8 eltwise add

test=develop

* - Fix to Michal PR

* - Fix

test=develop

* - Lint fixes

test=develop

* - Added checking if elementwise_mul can be used

test=develop

* - Added attribs to skip_attrs_set

test=develop

* - Improved broadcasting

test=develop

- fixes to compilation

- fix

- fix

- Lint fixes

test=develop

* - removed redundant condition

test=develop
Co-authored-by: NMichal Gallus <michal.gallus@intel.com>

* The arg of append() can be not Tensor temporarily.

* Add Seq2Seq as ProgramTranslator Unit Test. 

* set dtype of vocab_size_tensor to int64 to pass Windows-CI. 

* fix bug of prelu when rank not equal 4, test=develop

* fix prelu inference, test=develop

* fix api, test=develop

* fix shape when mode is chennel, test=develop

* remove debug code, test=develop

* add unittest, test=develop

As the title

* Add a StatValue class in the backend to represent a stat.
* Add a singleton StatRegistry to maintain the collection of stats.
* For the sake of code neatness, we only support type of int and float, which can cover most of the scenarios.

Some big models can timeout on Windows CPU machine. I added some timeout properties.

Add TSM as ProgramTranslator Unit Test. The TSM code is referred from PaddlePaddle/models#4229

* add MobileNet unittest test=develop

* fix cudnn random test=develop

* fix sync barrier with barrier monitor, test=develop

* enable load_program_state run in imperative mode; test=develop

* remove useless code; test=develop

[Dy2Static]Convert var.shape stmt and Convert the return variables of Tensor-dependent 'if' staments to Tensor if it not  (#24911)

* Support int and long: int or long -> six.integer_types. 

* Modify test_tensor_shape: fix bug and modify comment. 

* Support convert_var_shape to convert var.shape stmt

* Modify code in ifelse_simple_func.py because don't support return non-Tensor in Tensor-dependent 'if' stament currently. 

* Convert the return variables of Tensor-dependent 'if' staments to Tensor if it not. test=develop

[Dy2stat] decrease the batch size to decrease GPU usage.

* Move function 'convert_len' to file convert_operators.py 

* Support that for statements are transformed to while statements.

* Fix bug: raise None -> return None. 

* Support variable loaded and created in loop.

* Use int64 in Py2 and Py3 in function to_static_variable.

Fix the bug for elementwise_div op, when the first var is scalar; Use the shape 1 replace the -1 in shape.

[Dy2stat] Add word2vec as unittest

Add the support the 5d,6d tensor support for the reduce ops;
Add the same time, the compile time, it was 22 minutes, it was 21 minutes after fixed.

test=develop

* Support LoDTensorArray in reverse_op test=develop

* polish en doc and unittest code test=develop

* refine sample code test=develop

* add example of LoDTensorArray test=develop

* fix typo test=develop

* refine err_msg of pybind.cc, test=develop

* refine err_msg in tensor_py.h, test=develop

* refine error msg, test=develop

* fix test_exception, test=develop

* follow comments, test=develop

* cast var in convert_logical_XX. 

* Add convert_ifelse function in convert_operators.py  

* Add logical_transformer. Remove LogicalTransformer from loop_transformer.py 

* Revert modified tests in PR24799(convert_while_stmt). 

* Comment and modify code that doesn't support `return` statement. 

* Remove unnecessary class: MergeAssignTransformer, NodeTestTransformer and IfConditionVisitor in ifelse_transformer. 

* support selectedrows allreduce in multi-cards dygraph, test=develop

* remove useless import modules in unittests, test=develop

* add nccl cmake to get nccl version, test=develop

* add if-condition to compiled correctly, test=develop

* add detail version parseing for old nccl, test=develop

* polish camke details, test=develop

* fix remove test cmake error, test=develop

* fix cmake condition, test=develop

* change unittest camke list, test=develop

* fix unittest cmake rule, test=develop, test=framep0

The random failure at Windows may due to some random gt_boxes can cause some numbers in YoloV3 to be negative thus access the invalid memory. This PR tries to solve it.

Add Yolov3 as ProgramTranslator Unit Test. The YoloV3 code is referred from PaddlePaddle/models/dygraph/yolov3

* add the support of device index for device_guard.

* add mock barrier all
test=develop

* fix
test=develop

* fix
test=develop

* fix
test=develop

* add queue_generator_op, dequeue_op, enqueue_op and ut, test=develop

* Fix the field length in LoD scenario
* Fix the missed lod info when copy tensor in dump field
* Add some log to make debug easy

* add amp_check_finite_and_scale op, test=develop

* add cpu kernel, test=develop

* use bool, test=develop

* follow comments, test=develop

* fix bilinear initializer, test=develop