* Support optional residual add in fused_attention and fused_feedforward.

* Add checkpoint and add the check of add_residual when pre_layer_norm is false.

* Add TODO and change the python api to add add_residual argument.

* [MLU]: add shape kernel

* [MLU]: set output from cpu to mlu in shape kernel

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fl-ps v1.0

* .

* support N + N mode

* .

* .

* .

* .

* delete print

* .

* .

* .

* .

* fix bug

* .

* .

* shuffle_channel

* shuffle

* shuffle

* shuffle

* add bilinear_interp_v2 converter

* make device_id=current_id when not given

* use tracer to get current device id

* add activation layer

* Load models and run PE

* Stash all code

* Fix C++ UT and refine code

* Fix string length error

* skip windows ci because windows do not have wget

* Just print out in layer_test because bce param file changed

* Refine code

* Fix pdiparam download error for layer_test

* Polish FunctionSchema and rename some arg

* New Exector only once

* Add value type for ivalue

* Read params from one pdiparam file

* Replace ivalue using Variable

* Remove ivalue file

* Fix inner_exe new

* Using DeserializeFromStream to load tensor

* Split .cc files

* Polish cmakelist

* [CustomKernel] add custom kernel c api

* update

* update

* fix unable to export capi
Co-authored-by: Nronny1996 <524019753@qq.com>

* add squeeze2

* add squeeze

* add squeeze2,unsqueeze2

* merge develop

* fix format

* add conditions for squeeze2 and unsqueeze in op_teller

* merge develop

* add squeeze unsqueeze

* add squeeze unsqueeze

* add squeeze unsqueeze

* remove unsqueeze2_eltwise_fuse_pass

* add squeeze/unsqueeze

* add unary

* Support disable GC for some vars in standalone executor

* Setting skip_gc_vars in interprecore construction

* lazy creating work queue

* fix dry_run

* Optimize prod's python implementation for dygraph.

* Change key_dim to head_dim.

* Add comment in unittest.

* Disable TF32 in unittest.

* rename yaml file

* fix merge conflict

* fix infrt

* add some kernel(csr*dense->csr, dense*dense->csr) of SparseTensor matmul

* fix CI

* fix CI

* fix comment

* fix comment

* Refactor port.h

* Remove some unnecessary code

* Fix CI errors

* fix starganv2

* fix starganv2 stop_gradient end error

* fix edvr_starganv2

* fix mul kernel to fix optional ddx

* fix typo

* Support sequential run for standalone executor

* Add UTs

* Fix test_standalone_multiply_write

* Remove unnecessary UTs

* [MLU]: add elementwise_max mlu kernel

* [MLU]: add int32 support for elementwise maxk MLU kernel