* support shard reader

* support shard reader

* add parallel mode

* update process mesh

* add method to compute comm_group

* implement dist_embedding forward func

* implement dist matmul forward func

* implement dist reshape forward func

* add transpiler framework

* add transpiler forward

* implement transpiler forward

* implement transpiler backward & update

* add process

* add unitest

* chmod

* chmod

* chmod

* update unitest

* add unitest for gpt

* remove unused print

* rename transpiler --> partitioner

* rename transpiler --> partitioner

* chmod

* chmod

* bug fixed

* remove amp function

* update case for dp mode

* update case for dp mode

* aded slice FWD FP32

* added tests for slice FWD FP32

* added slice bwd

* added bf16 tests

* CI fix

* CI fix

* added reason to skip_if

* minor change

* temporary fix for failing test

* temporary fix

* changes after review

* CI rerun

* merge dense

* log level

* tensor copy sync

* format

* add cache for send_recv

* add eval_batch for pipeline

* add eval batch for pipelineparallel

* add style code

* modify fetch logic, use D2H Stream, test=develop

* refine, test=develop

* add strided_slice_grad op for npu

* support setting linewith when printing tensor

* fix ut

* refine code

* update comments

* use small precision since windows/linux has different ramdom value

* fix typo

* adjust parameter order for consistency

* add input and output docs for vision transform

* modify dy2stat error message in compile time

* fix variable name

* fix bug:When axes in paddle.sile is a tuple, an error occurs.

* polish code.

* stablize depthwise conv

* clean commend

* support KL label smooth

* update UT for KL label_smooth

* add ElementwiseTernary, Reduce, ReadDataStride

* Support append method and initialized value for List in ControlFlow

* polish error msg and en doc

* fix code style

* Support getitem by Bool index

* delete some debug info of bool index

* support the case that the shape of bool index is different from indexed tensor

* support setitem by bool index

* add the unittest for throwing exception

* merge conflict

* add check for int tensor when index is bool

* polish code

* fix unittest on windows

* refine pybind interface

* support statistic MemSize of AllocatorPool

* Replace mutex into atomic

This PR adds fused transformer python related files. It defines interface of fused transformer.

Fused transformer implements an optimized version of transformer layer (in python/paddle/nn/layer/transformer.py). In this PR, four layers (functions) are defined:
(1) FusedMultiHeadAttention: multi-head attention layer
(2) FusedFeedForward: feed forward layer
(3) FusedTransformerEncoderLayer: transformer encoder layer
(4) FusedTransformer: transformer layer

* Add model for ResNet50 for Dy2stat AMP training

* fix timeout

* fix dataloader

* [NPU] fix cmake for ascend ci, test=develop

* update paddle_build.sh scripts, test=allcase

* Revert "Revert "Add copy from tensor (#34406)" (#35173)"

This reverts commit 32c1ec42.

* add template instantiation

[Background]
Expansion in code size can be irreversible in the long run, leading to huge release packages which
not only hampers user experience but also exceeds a hard limit of pypi.

In such, NV_FATBIN section takes up 86% of the compiled dylib size, owing to the vast number of GPU
arches supported.

This PR aims to prune this NV_FATBIN.

[Solution]
In the new release strategy, two types of whl packages will be involved:

Cubin PIP package:
PIP package maintains a smaller window for GPU arches support, containing
sm_60, sm_70, sm_75, sm_80 cubins, covering Pascal - Ampere arches

JIT release package:
This is a backup for Cubin PIP package, containing compute_35, compute_50, compute_60,
compute_70, compute_75, compute_80, with best performance and GPU arches coverage.

However, it takes around 10 min to install due to the JIT compilation.

[How to use]
The new release strategy is disabled by default.
To compile for Cubin PIP package, add this to cmake: -DCUBIN_RELEASE_PIP
To compile for JIT release package, add this to cmake: -DJIT_RELEASE_WHL

* notest;test=cpu

* test

* test=document_fix

* add trt error information.

* rerun ci

* fix CI skip cc test error, test=develop

* remove test code, test=develop

As the title, see details at the PR description.