* fix dist train reduce mode

* fix previous fix

* fix dist transpile with memopt

* update api.spec

* polish dist transpile api

* update apispec

* update apispec

* refine dist train RPC deps

* clean up

* clean up

* fix ut

* remove input for fetch_barrier

* follow comments

(cherry-pick from commit 478eeabd)

(cherry-pick from commit 3de45566)

Merge pull request from #12843 from NHZlX:fix_ssa_bug_for_trt

Because of the changing trt engine op from gpu to gpu mode.

(cherry-pick from commit 145b20c160329428c1ec05c94287708aa1ed6f63)

cherry-pick from commit 133ec696

(cherry picked from commit 1600ba86)

fix ProgramToGraph

(cherry picked from commit e525aa23)

* fix auc layer and add check for auc op

* use input to check if states are inited

* optimize code

* dist transpiler add control dependency var between send and recv

* fix async deps

* follow comments and refine

* fix deps connect for rpc ops

Add dependency to send recv

* Disable in_place in batch_norm API.

* add rw lock test

* optimize read_write and wirte_read test

Optimize selected rows for dist lookup table with rwlock 

This reverts commit bf3c3496.

* Remove Data Sharing between input and output in scatter_op

* Removed data sharing in backward op

* "cherry picked operators changes"

* "remove duplicated code"

* "add constant setter"

* "add get expected kernel"

* "fix ci"

* "add fill constant"

* Add three modes for prelu_op.