* fix some matmul that X and Y both other ops outputs, do not dequantize the Y.

* fix CI format

* fix according to review

* add nproc_per_node for DistributedFusedLamb

* fix nproc_per_node communicator bug

* fix ring_id = 1 init bug

* fix ci

* fix test_parallel_executor_mnist.py

* fix sign-compare warning

* fix sign-compare 2

Co-authored-by: NLiu Yiqun <liuyiqun01@baidu.com>

* fix pylayer forward output code

* refine

* move_group_norm

* move group norm backward

* fix code format

* modify code according comment

* support matmulv2 in multihead

* Polish codes and memory usage for fused_gate_attention.

* Fix wrong reduce_dims in fused_gate_attention when computing gradient of nonbatched_bias.

* optimize dump_to_cpu for dymf

* code clean. test=develop

* fix func. test=develop

* fix code style. test=develop

* fix. test=develop

* add method for post ops

* format code

* change post-ops pattern

* code style

* add use_master_acc_grad

* add ut

* Update Profiler

* make HostEventRecorder templated

* fix bugs of reducer

* update

* update

* [MLU]support cast double type

* [MLU]fix cast test

Transpose optimization with assitant of  Chengdu Supercomputing Center and auto_tune operation (#42704)