* [Auto Parallel] Move the distributed info from python to c++

* [Auto Parallel] Add dist_attrs for VarDesc and OpDesc

* [Auto Parallel] Add the lost file

* [Auto Parallel] Make the dist attr be unique_ptr

* [Auto Parallel] Add the proto conversion

* [Auto Parallel] Improve the proto support

* [Auto Parallel] Fix the bugs for adding a device or a link

* [Auto Parallel] Add the C++ ProcessMesh and DistributedMapper

* [Auto Parallel] Improve the impl of these dist attrs

* [Auto Parallel] Pybind11 ProcessMesh and DeviceMesh

* [Auto Parallel] Fix the unittest problem

* [Auto Parallel] Explicitly add the src file for auto_parallel target

* [Auto Parallel] Add the proto depedency explicitly

* [Auto Parallel] Fix the cmake bug on windows and mac

* [Auto Parallel] Remove the pybind11 header file in process_mesh.h

* [Auto Parallel] Remove unused codes

* [Auto Parallel] Check whether the dist attr is null

* [Auto Parallel] Implement the assign operator for OpDesc explicitly

* fix proto bug

* add ut

* reset need_update for var_desc

* refine code

* fix var desc order issue

* fix bug of std::move and others

* fix an compile error in debug mode

* fix wrong copy assignment operator
Signed-off-by: Ntiancaishaonvjituizi <452565578@qq.com>

* reformat
Signed-off-by: Ntiancaishaonvjituizi <452565578@qq.com>

* reformat
Signed-off-by: Ntiancaishaonvjituizi <452565578@qq.com>

* fix ArrayRef constructor following llvm

* fix format

* fix conflict with master

* [Auto parallel] Make the id of var and op unique

* [Auto Parallel] Rename back dist_context to distop_context

* add new API: paddle.clone;Tensor.element_size;nn.utils.parameters_to_vector

* fix comment

* add const OpDesc id()

* add const for VarDesc::id()

* add auto_parallel dir

* mv to paddle.distributed

* add shard_xx api

* add distributed attrs for var

* add ut, test=develop

* add dist

* update

* update

* update

* update

* update

* update, test=develop

* update, test=develop

* update, test=develop

* update, test=develop

* update, test=develop

* update, test=develop

* update, test=develop

* update

* update

* update

* update

* update

* update, test=develop

* update, test=develop

* update

* update

* delete unused proto

* resotre op_desc

* restore type_defs

* update var_desc

* remove dimss_mapping for proto_pybind

* update interface.py

* update framework.py

* update

* update

* add auto_parallel dir

* mv to paddle.distributed

* add shard_xx api

* add distributed attrs for var

* add ut, test=develop

* [WIP] Add the auto completion feature and related codes

* [WIP] Improve the auto completion and related codes

* [WIP] Make the auto completion to support data-parallel

* [WIP] Make the completion support mp and dp+mp

* [WIP] Refactor auto completion unit test for MLP

* [WIP] Refactor the implementation of DistributedOperatorImpl

* [WIP] Improve dims_mapping update rule and fix a bug

* [WIP] Support auto completion for one transformer decoder layer

* [WIP] Add a minor change

* [WIP] Fix a bug within the uint test

* Shard XShape tensor, add embedding completion and refactor code

* Add the distributed_operators dir to setup.py.in

* Improve the completion process and add the unittest for gpt

* fix process_mesh ut

* fix process_mesh ut

* update

* update, test=develop

* Add support for automatically completing distributed attrs of special ops

* update

* update

* update

* fix doc sample codes, test=develop

* improve coverage, test=develop

* add static_mode check, test=develop

* Model the cluster for cost model and physical mapping

* update, test=develop

* add set_placement, test=develop

* Add the check to make sure the candidate tensors' size is great than zero

* update doc, test=develop

* update doc, test=develop

* update doc, test=develop

* update doc, test=develop

* update, test=develop

* Auto mark dist attrs annotated by user

* update ndarray to nested list, test=develop

* update, test=develop

* Add auto-completion module for auto-parallel (based on PR#33804)

* Remove unnecessary files

* Remove unrelated files for the auto completion pr

* Update the unit test to improve the coverage

* Modify codes based on reviews

* Minor changes for CI

* Improve some codes based on new comments

* Fix bugs caused by shallow copy in attributes.py
* Imporve amend_distributed_attr_for_program in context.py
* Other changes for weihang's comments
Co-authored-by: Nsandyhouse <lilong12@baidu.com>

* add auto_parallel apis

This PR added optional boolean is_parameter and stop_gradient in the VarDesc proto, and remove them during save_inference_model

* use iwyu clean include, test=develop, test=win

* compilation error, test=develop

* fix compilation error2, test=develop

* fix compilation error3, test=develop

* fix compilation error4, test=develop

* fix compilation error5, test=develop

* fix compilation error6, test=develop

* fix compilation error7, test=develop

* fix compilation error8, test=develop

* fix compilation error8, test=develop

* fix compilation error10, test=develop

* fix compilation error11, test=develop

The new "fluid.data" changes old "fluid.layers.data":

1. Add shape and dtype check.
2. Remove "append_batch_size" parameter. We won't offer this in the new data layer because other deep learning platforms don't have this kind of data layer pre-processing. It may confuse users.
3. Remove "stop gradient" parameter because the data layer doesn't do back-propagation

TODO：
Now data layer feeded by executor is checked, will we want to check the feed data of readers in the future?

test=develop

test=develop

test=develop

test=develop

test=develop

* Adding set_capacity method support

* Adding Python for make_channel

* Updating notest_concurrency

* Write python for make_channel method

* Write python for make_channel method

* Fix make_channel and test

* Placeholder ops for channel send, recv and close

* Adding ToTypeIndex method to var_type.h

* Add var_type.h to channel:

* Added POD_Type to the method

* Add CHANNEL to executor

* Updated get and set DataType to accomodate Channels

* Updating get and set to incorporate channels

* Adding CHANNEL as supported VarType in protobuf

* Removing unecessary import

* Fixing VarDesc to adapt to Channel as VarType

* Add channel.h to executor

* Remove innclude from channel

* Updated var_type to support Channel as  var type

* Adding get_channel to pybind

* Added ChannelHolder

* Adding make_channel as an op

* Adding ChannelHolder in channel

* Fixing typo

* Commenting out operators in concurrency

* Removing totypeid right now since we don't need it.

* Reverting python changes

* Fixing typo in framework.py

* Modify comments for ReaderHolder

* Move Pod Types from DataType enum to Type enum

* Fixed data_type.h

* Fix type in TensorDesc

* Add comment to framework.proto

* Fixed type in data_type.h

* Updated format of type in data_type.h

* Fix var_desc.h

* Fix op_kernel_type.h

* Fixed data_type_transform_test.cc

* Fix operator.h

* Fixed data_type_transform.cc

* Fixed op_kernel_type_test.cc

* Fix operator.cc

* Fixed data_layout_transform_test.cc

* Fix var_desc.cc

* Fixed assign_value_op.cc

* Fixed assign_value_op.h

* fixed protobuf.cc

* Fix data_layout_transform_test.cc and op_kernel_type_test.cc

* Fixed rnn_memory_helper_op.cc

* Fix progrma_desc_test.cc

* Fixed fill_constant_batch_size_like_op.cc

* Fix operator_test.cc

* Fixed fill_constant_op.cc

* Fixed gaussian_random_op.cc

* Fixed uniform_random_op.cc

* Fixed edit_distance_op.cc

* Fixed fill_constant_batch_size_like_op.cc

* Fixed rnn_memory_helper_op.cc

* Fixed chunk_eval_op.cc

* Fixed assign_value_op.cc

* Fixed assign_value_op.h

* Fixed cast_op.h

* Fixed cast_op.h

* Fix fill constant op

* Fixed clang for assign_value_op.cc

* Fix one_hot_op.h

* Fix one_hot_op.cc

* Fix fill_op.cc

* Fixed sum_op.cc

* Fixed sum_op clang

* Fix uniform_random_op.cc

* Fix gaussian_random_op.cc

* Fix backward.cc

* Fix protobuf.cc

* Fixed prune_test.cc

* Fixed op_registry_test.cc

* Fix data_device_transform_test.cu

* Fix travis error

* Fixed one_hot_op.cu

* Fixed op_registry_test.cc

* Fixed nccl_op.cc

* Fixing python tests

* Revert "Fixing python tests"

This reverts commit fccaa4c5.

* Fixing Pybind to remove data type

* Fixing tensor.py

* Updated the new files:

* Resolve error in merge conflict of fill_constant_batch_size_like_op.cc

* Refine Type system

* Fixing type inference

* Fixed create_reader_op.cc

* Fix var_desc.h

* Fixed executor.cc

* Fix shape_inference.h

* Fixed create_reader_op.cc

* Fix tensor_util.h

* Fixed var_type_inference_test.cc

* Fix shape_inference.cc

* Fixed sum_op.c

* Fixed read_op.cc

* Fix var_type.h

* Fixed beam_search_decode_op.cc

* sendrecvop_utils.cc

* Fix operator.cc

* Fixed lookup_table_op.cc

* Fixed op_desc.cc

* Fixed get_places_op.cc

* Fixed lod_rank_table_op.cc

* Fixed beam_search_op.cc

* Fix var_desc.cc

* Fixed lod_tensor_to_array_op.cc

* Fixed while_op.cc

* Fix program_desc_test.cc

* tensor_array_read_write_op.cc

* Fix assign_op.cc

* Fix executor.cc

* Fix protobuf.cc

* Fix protobuf.cc

Add a new type `Reader` for `VarDesc`, which can holds more than one
LoDTensor.

* limit variable type to lod tensor in memory optimization transpiler

* refine policy

* support while operator

* fix random seed and training data order

* refine get_cfgs method to support multi while operators

* refine codes

* Rename XXDescBind --> XXDesc

* Fix Compile

* Move framework.proto to proto namespace

* Fix compile

* Fix compile

* Fix Compile

* Add LoDRankTable

LoD Rank Table stores the `level` of `lod` which is ordered by sequence
length in descending order. It is useful when implement dynamic RNN and
is shared by dynamic RNN memory, dynamic RNN slice input and dynamic
RNN slice output operators.

* Add InferVarType

* compelete restoring program_bind from program_desc

* Fix bugs

* fix compile errors

* fix errors and add unit tests

* rename some vars

* Follow comments

* init

* ensure ids in lookup table op must be a column vector

* add book4 configuration in test_layers

* debug test_book4

* add test_word2vec

* follow comments

* follow comments

* add target to Backward, generate var in block when call backward

* modify backward_test

* fix executor_test

* set var desc default type to LOD_TENSOR

* update backward_test

* insert loss in the top level of backward

* create grad vars for all blocks in current program

* optimize code

* update test_program.py

* only create var for newly create blocks when backward

* Update VarDesc from design doc

* Fix GCC compile

* Fix unittest