* optimize update_loss_scaling_op by fused for loop to one kernel, test=develop

* remove useless while loop and optimize variable name, test=develop

* optimize variable name from out_addrs_tensor to out_addrs_mem, test=develop

* optimize variable name for readable by change prefix identifier from t_ to local_

* add alltoall api, test=develop

* support cuda11.2 and using gcc5.4 in cuda10.1

* fix manylinux py36 bug

* support cuda11.2

* fix python36 pip version problem in ubuntu

* save cuda11.0

* update 2.0 public api in nn

* replace Chinese character cause error in ci;synchronization with pr:#32588 to avoid 'ascii' codec in python2

* numbers used in paddle.nn.functional.norm but not imported

Co-authored-by: NXiaoguangHu <46782768+XiaoguangHu01@users.noreply.github.com>

* edit paddle.save/load API

* Update io.py

edit doc

* delete cpython-37.pyc

* Update io.py

edit doc

* Update io.py

recommit

* Update io.py

recommit

* Update io.py

recommit

* Update io.py

recommit

* clear 'BasicEngine' when an exception occurs in the backward.

* deal with conflict.

* deal with conflict.

* [OPs] Bug fix, fix the segment mean for illegal syncthreads usage.

* jit.save/load support function.

* delete unnittest test_jit_load_model_incomplete.

* edit code according to CI

* Modify the documentation.

* add note to doc.

* fixed docs.

* Fixed docs. test=document_fix

code bak.

fixed docs. test=document_fix

* Revert to previous version of python/paddle/fluid/backward.py

* fixed bugs.

* test=document_fix. Fixed examples.

* fix cross_entropy calculation error

* add unittest and fix static

* UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1788: ordinal not in range(128)

test=document_fix

str(doc) in python2

test=document_fix

* update md5 function in count_api_without_core_ops.py

str in py2 is different.

test=document_fix

* Support list and tuple for parameters of layer_norm, multiprocess_reader, DatasetFolder and ImageFolder.

* add unittest for layer_norm.

* add require gpu for example.

This reverts commit 4b7242b0.

Co-authored-by: NYang Zhang <yangzhang@live.com>

* fix amp bug

* fix name of wordsize

* add sendrecv, test=develop

* clear CUDA compile environment on windows

* fix Windows CI

* fix Windows CI

* fix Windows CI

* new optimize for where_index_op with prefix sum version.

* write a scan prefix sum kernel with stream for where index op.

* optimize where_index by using cub::DeviceScan::InclusiveSum instead of imperfect self-kernel.

* remove CheckTrue struct and rename stide_array for readable.

* optimize variable name for readable.

* optimize function name and annotation.

* optimize pull sparse

* optimize pull sparse

* change macro

* format

* fix bn docs

* fix unittest

A follow up PR of #32420, we changed the doc of python/paddle/fluid/layers/tensor.py in that PR and we are changing python/paddle/tensor/creation.py in this PR.

* Modified params of some APIs to support tuple and list.

* fixed bug.

* Modified the return value of tensor.grad from numpy as tensor.

* Modify unittests.

* fixed bugs.

* Add warning info for x.grad

* fixed unittests which used x.grad

* fixed bug.

* fix dataloader exit error if user exit program when dataloader is still iterating. test=develop

* Unset ReserveSpace for inference program.

* Support training from an inference program.

* change prepend to append

* fix ut

* add testcase

* fix ut

* fix test_custom_relu_model