* replace amp.use_pure_fp16 with amp.dtype and amp.level

* old api still use use_pure_fp16

* test_fuse_adamw_pass still use use_pure_fp16

* add test case tuning recompute with amp(float16,o2)

* reset new test case properties TIMEOUT 60

* set smaller value of batch_size and batch_num

* deepcopy dist_context fix _rename_input problem

* fix loss name after cast

* set tuning.enable=True and use engine._tune()

* restore some changes in _rename_input()/_rename_output()

* add self.amp_dtype for _cast_loss() in auto_parallel_amp.py

* fix insert op index in _cast_loss()

[xdoctest][task 184-185] reformat example code with google style in `distributed/auto_parallel/static/*` (#56666)

* [Doctest]fix No.184,185, test=docs_preview

* add env skip

* fix @staticmethod

* fix

* add xdoctest for v2

* fix

* for verify

fluid operator support new comm library

* u

* u

* u

* compatiable new comm library upgrade for c_allgather, c_reduce, c_reduce_scatter and c_scatter.

* Remove useless comments in process_group.py

* Polish code style.

* Fix some problems.

* Remove use fluid api in phi comm_context_manager.

* Add PPADDLE_WITH_CUDA and PADDLE_WITH_NCCL micro judgement.

* Fix bug of HIP architecture.

* Fix some problems.
1. remove useless loggings.
2. Fix conditional compilation for HIP.
3. Fix problems of test_pass_generation_pipeline.py. It calls paddle.distributed.init_parallel_env() at first,
then auto.Engine calls _init_comm(), which will calls process_group.instantiate(). However, init_parallel_env() will call
paddle.distributed.barrier(), it will call CreateNCCLEnvCache and create corresponding NCCLCommContext. But dev_id is not
set, as a result, NCCLCommContext's dev_ctx is not initialized.

* Fix some problems.

* Polish code.

* Polish code.

* Revert compatiable upgrade for communication operators. Their upgrades
will be submitted in another PR.

* Remove StaticTCPStore.

* Remove useless modification.

* Remove useless set_cuda_device_id.

* Polish code.

* Remove fluid header files in phi files.

* Remove useless comments.

* Fix problems of hip arch.

* Fix some problems.

* Polish code.

* Polish code style.

---------
Co-authored-by: hitywt <yuwentao126@126.com>

[xdoctest][task 181-183] reformat example code with google style in `sparse/multiary.py`,`distributed/auto_parallel/*` (#56665)

* [Doctest]fix No.181-183, test=docs_preview

* add env skip

* fix eval_loss bug in parallelizer_v2.py

* fix fetch problem in pass_utils.py

* modify feed_data for dataloader in pipline parallel mode

* add pre-commit

* remove read op, use feed op

* fix validate batch_size

* tiny fix

* support catch EOFException

* fix conflict

* fix conflict

* fix executor if cond

---------
Co-authored-by: Frida-a <2624653516@qq.com>

* shard_tensor support static graph

* add comments

* add dy2static ut

* use property in c++ side

* fix: updated code examples.

* fix: added paddle.seed

* fix: updated code style

* Apply suggestions from code review

* refactor: refine detail of code examples

* Update python/paddle/distributed/auto_parallel/static/process_mesh_v2.py

* fix: refine detail

* fix: refine detail

* Update python/paddle/distributed/auto_parallel/static/process_mesh_v2.py
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* refactor: refine detail

* refactor: refine detail

* fix: refine doc

---------
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* make params_grads order same bewteen dynamic and static mode

* revert inplace clip

* use sorted attribute to control

* tiny fix

* fix find loss_grad_op

* remove the optimizer base and learning rate base

* fix bug

* fix bug

* open

* update

* Update autoparallel DistributedDataLoader

* add places for engine.dataloder()

* simple reaplce

* for debug

* fix bugs

* fix some bugs

* del fill_constant_batch_size_like

* add shard tensor api

* add DistAttr api

* add unittest for coverage

* fix process mesh sample code

* fix checking error

* Support nvprof for auto parallel

* Fix CI errors

* Fix CI errors

* remove allreduce before c_allgather

* update reshard insert_fill_constant_op func

* insert_fill_constant_op add shape arg

* add skip_gc_vars for 1f1b schedule mode

* add pp_degree and pp_stage

* base rule

* add sharidng merge

* add sharidng axis merge

* define unified data class for inferencing dist_attr

* test wrap DistTensorSpec in dygraph mode

* matmul main logic done

* define unified data class for inferencing dist_attr

---------
Co-authored-by: NYichen Zhang <zhangyichen03@baidu.com>

* auto parallel support pipeline scheduler with standalone executor

* rm check_fetch

* update cmakelist and flags env

* rm set micro batch id

* rm import

* update utils func

* raise error when merge tensor for return_numpy is False

* fix _pipeline_opt

* fix unittest

* bump ruff to 0.0.271 and update config

* exclude third_party

* bump ruff to 0.0.272

* refine config

* eager call all2all to avoid p2p hang in lazy init

* update

* [AutoParallel] add 1F1B

* rm amp

* [AutoParallel] update while control_flow with pipeline

* update process group instantiate

* fix micro_bsz for reshard

* update api for micro batch size

* add strategy for dp optimization

* [Auto Parallel] Reorganize the fold structure

* [Auto Parallel] Fix some import errors

* global view process_group

* fix import

* fix attr

* fix tunner init comm

* [CustomDevice] fix auto_paralell

* update

* update

* update

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* fixed cyclic reference that caused patial import

* fixed bad change

* fix bad import

* fix bad import

* fix bad import

* fix ut failed caused by change in_dynamic_mode

* fix ut failed caused by change in_dynamic_mode

* fixed usage of in_dynamic_mode() or in_dygraph_mode()

* revert python3 to python in .pre-commit-config.yaml

* fix merge conflicts

* rm cmake npu

* Update generic.cmake

* Update generic.cmake

* rm npu

* rm use_npu

* rm npuid

* rm use_npu

* rm npuid

* delete npupinned

* roll back sth.

* roll back sth.

* delete npupinned

* roll back sth.

* roll back sth.

* rm npu

* rollback something

* rollback npu identity

* rollback npu identity

* move DataLoader to paddle.io. test=develop