[Kunlun]Multi xpu dygraph performance optimization , add distributed.spawn support for multi xpu and some bug-fixes (#31130)

* Add support for mkldnn ops types selection with FLAGS in dygraph

* use regex to match DNNL verbose

* python3 encoding fix

* support use add instead of sum to do gradient accumulation

* add inplace addto pass

* add grad_add op and inplace addto pass

* remove debug code

* code refine

* fix bug when sereral sum ops inserts at same op_idx

* fix Flags type

* add addto attribute for conv3d

* fix ut

* code clean

* fix type

* get use of global 'use_mkldnn' in layer_helper

* update for CI

* update for CI, relu test

* update for CI, relu test added, make FLAGS_use_mkldnn a public flag

* added more strict tests, fixes after review

* fixes after review

* fixes after review, CI stuff

* update the demo code and the doc of varbase.backward.

* update the doc of the fake interface `paddle.fluid.Variable`.

* remove BackwardStrategy.

* add flags_call_stack_level

* update

* refine code

* Remove the NGraph engine from PDPD repository
1. Each operator was removed from the operator's directory
2. Each test was removed from the unittest directory
3. The parallel executor support was removed from the PDPD
4. The CMake file was removed from the PDPD
5. The NG flags were removed from the repository
test=develop

* Remove ngraph from:
1. Cmake file
2. Python file
test=develop

* Add a function to update FLAGS

test=develop

* Add a function to update FLAGS

test=develop

* expr flags

* Add a function to update FLAGS

test=develop

* distinguish public/private vars, test=develop

* fix windows issues, test=develop

* expr flag

* Add functions to get and set FLAGS

test=develop

* Add functions to get and set FLAGS

test=develop

* Add functions to get and set FLAGS

test=develop

* Add functions to get and set flags

test=develop

* Add functions to get and set FLAGS

test=develop

* Add a function to update FLAGS

test=develop

* Add a function to update FLAGS

test=develop

* Add functions to get and set flags in Paddle

test=develop
Co-authored-by: Nsneaxiy <sneaxiy@126.com>

* distinguish public/private vars, test=develop

* fix windows issues, test=develop

* add double grad implementation for dygraph, test=develop

* polish code, add uts, test=develop

* fix place bug, test=develop

* polish codes, add more uts for coverages, test=develop

* add no_grad_set, test=develop

* add star gan ut, test=develop

* follow comments, test=develop

* add recorded cuda memory apis, fix typo, test=develop

* add more ut, test=develop

* follow comments, test=develop

* fix py35 incompatible issues, test=develop

* update ScopeBufferedSSAGraphExecutor&AsyncSSAGraphExecutor&ThreadedSSAGraphExecutor&FastThreadedSSAGraphExecutor&ParallelSSAGraphExecutor&ParallelExecutor for fetching unmerged results.

* add the unit test for fetch_unmerged.

* update ut for multi-card and multi-cpu.

* add the error message and the user suggestion in FetchOpHandle. test=develop

* add some debug flags to auto growth allocator, test=develop

* add comments about auto growth, test=develop

* use system allocator in unittests, test=develop

* fix op bugs, test=develop

* fix tensor copy bug when src and dst are the same, test=develop

* add global value getter setter, test=develop

* fix error messages, test=develop