* refine cudnn softmax

update the doc for the some ops, ceil asin, atan

* Support Heter Parameter Server

* add doc; notest

* fix doc; notest

* update doc; notest

* refine optimizer && adam

* refine optimizer; notest

* add adam

* fix doc

* fix doc && add adamw; notest

* add error message

* bug fix

* refine rmsprop && adamax

* fix ci

* buf fix

* update comment

* unify arguments place; notest

* fix ut, test=develop

* bug fix

* fix conflicts, test=develop

* add examples code

* bug fix

* fix comments

* fix sample code

* add sample code for Optimizer

* add adamax ut, test=develop

* fix rmsprop ut, test=develop

* add ut for optimizer.py and adamw.py

* first commit of adadelta optimizer

* fix learning rate

* fix adadelta doc and add sgd momentum

* remove unused fluid

* fix codestyle

* Update test_adam_op.py

* Update test_adam_op.py

* fix SGD in 2 unittests

* fix SGD in 2 unittests

* fix ci

* fix ut
Co-authored-by: NMRXLT <xlt2024@gmail.com>
Co-authored-by: Nmapingshuo <mps2012@yeah.net>

* fix bug

* add version check

* fix docs, test=document_fix

* fix formula, test=document_fix

* decouple weights and bias from fc primitive in cache

* removed reduntant update of pointers

* refine manual seed

* fix ci problem

* fix unittests

* fix unittest

* set is_init_py=false in manual_seed

* fix unittest

* fix bernoulli_op

* fix(unittest): change random_seed to manual_seed

* 🐞fix(unittest): fix manual_seed

* trigger ci

* fix test_sentiment

* fix test_imperative_save_load

* fix test_uniform_random_op

* fix test_uniform_random_op

* fix test_jit_save_load

* merge develop

* fix manual_seed

* fix manual_seed

* use global engine

* use shared_ptr

* fix double free

* fix bug

* fix bug

* fix bug

* fix test bug

* fix test bug

* fix test bug

* fix ci

* fix remainder, floordiv

register cumsum Op version for compatible Op upgrades

add broadcast feature for elementwise logical op

* Fix clip input check

* Fix default min/max value

* Allow both max and min to be None

* Register op change

* Revert OP signature change

As the title

* add intepolte_v2

* fix linear interp

* polish unittest, test=develop

* update code samples to 2.0 API, test=develop

* remove warning, test_develop

* add name in attrs, test=develop

* polish code, test=develop

* change Align to align, test=develop

* fix unittest in py3,test=develop

* fix coverage, test=develop

* fix coverage, test=develop

* fix for windows ci, test=develop

* fix coverage, test=develop

* update doc, test=document_fix

* add attr(dtype)

* refine code

add collective op for cpu using gloo and paddle.distributed.* apis

add the top v2 for the paddlepaddle api 2.0

test=develop

* add unique_v2 op

* remove unique_v2 op

* update doc

test=develop

* Enhance affine grid operator:
1. Add cuda kernel
2. Add align corners options
test=develop

* Move new affine_grid api to functional
test=develop

* Add CUDA kernel for affine_grid.
test=develop

* Add more unitest for grid sample API
test=develop

test=develop

This PR enhance CPU kernel and add new CUDA kernel to make grid_sample support:

- align_corners: with bool type.
- padding mode: which can be in ['zeros', 'reflect', 'border']
- Interpolation mode: which ca be in ['bilinear', 'nearest']

The old CPU and CUDNN version only support align_corners=true, padding_mode='zeros' and interpolation_mode='bilinear'.

The behavior of the new version op in default mode is compatible with the old version.

change to use bce_loss op, add numel check for bce_loss.

* add div, floor_div, remainder

add the isnan, isfinite, isinf api for the paddle 2.0