* Add solutions to PyLayer which is unsupported in DataParallel

* modify note format for parallel.py

* modify docs of dataparallel

* add docs of dp with pylayer

* modify docs format

* modify example format

* change example of dp with pylayer

* add unittest for dp with pylayer

* modify ut

* merge latest codes

* update

* modify for CI-Coverage

* modify text-indent

Intergrate StandaloneExecutor in Static.Executor Interface with FLAGS_USE_STANDALONE_EXECUTOR (#35628)

* Intergrate StandaloneExecutor in Static.Executor Interface with FLAGS_USE_STANDALONE_EXECUTOR

* Enhance unittest and clean code in StandaloneExecutor

* polish unittest

* dy2stat_error: add revise suggestion for two error cases

* fix test_error

* fix review

add fill_ backward

* fix raw optimizer gm

* update

* update ut

* Add empty_cache api to release idle gpu memory hold by allocator,test=develop

* Add empty_cache api to release idle gpu memory hold by allocator,test=develop

* Add empty_cache api to release idle gpu memory hold by allocator,test=develop

* Fix test coverage problem for empty_cache

* delete redundant check for empty_cache

* fix the problem of empty_cache's doc

* delete the nvidia-smi comment in doc of empty_cache, test=document_fix

* Change uts to nightly mode

* remove test_trt_pool_op from parallel_UT_rule.py,test=document_fix

* fix interpolate launch error, test=develop

* fix area mode for interp, test=develop

* fix instance norm index error

* add unittest

* update

* fix

* fix cumprod docs

* fix cumprod op docs; test=document_fix

* [RC22] Fix linear with matmul_op replace

* [RC22] Fix linear with matmul_op replace

* [RC22] Fix linear with matmul_op replace

* [RC22] Fix linear with matmul_op replace

* [RC22] Fix linear with matmul_op replace

* add flatten/flatten2 converter test cases

* add fatten/flatten2 trt converter test cases

* reshape support zero-input

* add unitest

* revise error message

* upload global scatter and global gather operators related files

* fix github name

* fix CI error

* fix review and CI error

* fix inf,nan error and modify unittest samples

* add unittest samples

* add unittest samples

* fix unittest error

* test=document_fix

* test=document_fix

* modify doc and add unittest samples

* fix error newline in constant

* modify doc after mentor review

* modify __all__ and doc

* modify doc

* support grad group

* fix single card condition

* add group_norm trt converter test case

* update group_norm trt converter test case

This reverts commit ae93d9c2.

* support hybrid parallel inference helper class

* [ROCM] fix top_k_v2 with large shape

* [ROCM] fix top_k_v2 with large shape

* implemented clip op bf16/fp32

* added skipping if not cpu or bf16

* CI rerun after bf16 package change

* added parentheses to ensure formatting