* add code

* add unit test

* Revert "[GPUPS]Optimize dymf kernel (#43911)"

* support scope_guard

* fix test

* merge and add base support for non-local for

* for and while non-local support

* fix ci errors: v1

* fix bug

* fix

* fix code

* fix

* fix

* fix

* Remove boost::variant for FetchResultType

* Fix pybind errors

* modify graph_pattern to thread_local

* modify graph_pattern to thread_local

* Add new attr of fused_multi_transformer

* fix format

* add note

* add in layer

* fixfixfixfix

* add relu6 kernel and yaml

* format files

* format code and fix bug

* fix build failed

* support complexd selected_rows kernel in yaml

* support configuring optimizer api in yaml

* fix data transform bug

* fixed bug for pass & engine

* fixed bug for benchmark GPT-3

* Update the lock logic used in CinnCompiler::Compile.

* add comparisons trt converter

* separate variable scope and scope

* hot fix for lod_tensor_blocking_queue

* fix bug that variable exists in global scope

* inference add convert to mixed model ability.

* move cross form legacy_api.yaml to api.yaml

* move diagonal to api.yaml

* skip xpu conv2d fp16 unitest
*test=kunlun

* minor
*test=kunlun

* [Dy2Stat]Polish all API name of _jst

* add unittest for PR43688

* [UPDATE FLUID API] only reference in paddlerec

* change lr

* [UPDATE FLUID API] only reference in paddlerec

* update by reviews

* change to condition to find python interpreter to avoid skipping the find process.
PYTHONINTERP_FOUND is the best signal that python interpreter is found.

* Remove output arguments from functions.
Replace pointers with references

* Name used bool flags

* Reorder functions

* Enable bfloat16 data type

* Give declarations some space

* Style

* Style

1. test_parallel_executor_seresnext_base_gpu failed on 2 P100 GPUs with `470.82` driver.
```
======================================================================
FAIL: test_seresnext_with_learning_rate_decay (test_parallel_executor_seresnext_base_gpu.TestResnetGPU)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/paddle/paddle/build/python/paddle/fluid/tests/unittests/test_parallel_executor_seresnext_base_gpu.py", line 32, in test_seresnext_with_learning_rate_decay
    self._compare_result_with_origin_model(
  File "/opt/paddle/paddle/build/python/paddle/fluid/tests/unittests/seresnext_test_base.py", line 56, in _compare_result_with_origin_model
    self.assertAlmostEquals(
AssertionError: 6.8825445 != 6.882531 within 1e-05 delta (1.335144e-05 difference)
----------------------------------------------------------------------
```
2. To be more accuracte on evaluating loss convergence, we proposed to apply IOU as metric, instead of comparing first and last loss values.
3. As offline discussion, we also evaluated convergence on P100 and A100 in 1000 interations to make sure this UT have the same convergence property on both devices. The curves are showed below.
![A100-Single, P100-Single and Diff (1)](https://user-images.githubusercontent.com/13541238/175461920-25df6101-6dd8-4387-862c-d1c8e9299c57.png)