* fix raw optimizer gm

* update

* update ut

* Add empty_cache api to release idle gpu memory hold by allocator,test=develop

* Add empty_cache api to release idle gpu memory hold by allocator,test=develop

* Add empty_cache api to release idle gpu memory hold by allocator,test=develop

* Fix test coverage problem for empty_cache

* delete redundant check for empty_cache

* fix the problem of empty_cache's doc

* delete the nvidia-smi comment in doc of empty_cache, test=document_fix

* Change uts to nightly mode

* remove test_trt_pool_op from parallel_UT_rule.py,test=document_fix

* fix interpolate launch error, test=develop

* fix area mode for interp, test=develop

* fix instance norm index error

* add unittest

* update

* fix

* fix cumprod docs

* fix cumprod op docs; test=document_fix

* [RC22] Fix linear with matmul_op replace

* [RC22] Fix linear with matmul_op replace

* [RC22] Fix linear with matmul_op replace

* [RC22] Fix linear with matmul_op replace

* [RC22] Fix linear with matmul_op replace

* add flatten/flatten2 converter test cases

* add fatten/flatten2 trt converter test cases

* reshape support zero-input

* add unitest

* revise error message

* upload global scatter and global gather operators related files

* fix github name

* fix CI error

* fix review and CI error

* fix inf,nan error and modify unittest samples

* add unittest samples

* add unittest samples

* fix unittest error

* test=document_fix

* test=document_fix

* modify doc and add unittest samples

* fix error newline in constant

* modify doc after mentor review

* modify __all__ and doc

* modify doc

* support grad group

* fix single card condition

* add group_norm trt converter test case

* update group_norm trt converter test case

This reverts commit ae93d9c2.

* support hybrid parallel inference helper class

* [ROCM] fix top_k_v2 with large shape

* [ROCM] fix top_k_v2 with large shape

* implemented clip op bf16/fp32

* added skipping if not cpu or bf16

* CI rerun after bf16 package change

* added parentheses to ensure formatting

* change metaclass of Layer from pybind11_builtins.pybind11_type to type

* fix cast

* add ut

* sharding support dp

* sharding support mp

* sharding support pp