* Optimize prod's python implementation for dygraph.

* Change key_dim to head_dim.

* Add comment in unittest.

* Disable TF32 in unittest.

* Decoupling ipu sharding and modeling (#665)

* feat(shard): decoupling shard setting with modeling.

* fix(shard): split test cases to avoid failure.

* fix(shard): add function docs and fix typo.

* test(shard): add tests.

* test(shard): more test case.

* fix(): change ipu_index/stage default value to -1.

* fix format
Co-authored-by: Nczr-gc <96037699+czr-gc@users.noreply.github.com>

* set_state_dict not use state_dict hook

* add ut

* refine doc

* use tempfile to place temporary files

* update

* revert test_communicator

* fix test_dist_base

* rename yaml file

* fix merge conflict

* fix infrt

* add some kernel(csr*dense->csr, dense*dense->csr) of SparseTensor matmul

* fix CI

* fix CI

* fix comment

* fix comment

* Support sequential run for standalone executor

* Add UTs

* Fix test_standalone_multiply_write

* Remove unnecessary UTs

* [MLU]: add elementwise_max mlu kernel

* [MLU]: add int32 support for elementwise maxk MLU kernel

* Refine ifelse early return

* update paddle.distributed.launch

* add sample code

* update shell

* fix typo

* fix typo

* update docs

* rm code

* fix doc 2

* fix doc 3

* fix doc 4
Co-authored-by: Nroot <root@sgjur-pod004-1.ipu.graphcore.cn>

* change logic for optimize

* modifty

* optimize the backward speed of EinsumOp

* add cache optimizer for einsum op

* EinsumOp: fix new dygraph mode error

* fix bug

* change Cache->InnerCache

* fix code

* fix

* add nan inf utils for einsum op

* add as_extra

* memory optimizer for einsum

* update code

* fix the bug that _DataLoaderIterMultiProcess use time to generate the seed

* use np.random.randint to generate a base seed

* cmake-lint

* update

* Replace np.bool/np.bool8 with np.bool_

* Replace np.object with np.object_

* Replace np.complex with np.complex128

* Replace np.float with np.float64

* Replace np.int with np.int_

* Rerun pre-commit for newer pre-commit configuration

* Use builtin bool instead of np.bool_ based on the context

* fix fetch list

* fix unittest

* calculate_gain; test=document_fix

* Constant; test=document_fix

* KaimingNormal; test=document_fix

* KaimingUniform; test=document_fix

* randint; test=document_fix

* squeeze;test=document_fix

* argmin; test=document_fix

* argmin; test=document_fix

* triu; test=document_fix

* add_n;test=document_fix

* unique; test=document_fix

* topk; test=document_fix

* squeeze;test=document_fix

* randint;test=document_fix

* argmin; test=document_fix

* constant; test=document_fix

* constant; test=document_fix

* KaimingNormal; test=document_fix

* kaiming; test=document_fix

* unique; test=document_fix

* gpups default config and dataset

* codestyle

* add unittest

* code style

* 'TripletMarginDistanceLoss'

* 'test_file'

* '2022_03_27'

* 2022-03-31

* 2022-04-05

* 2

* 2022-04-17

* 2022-04-17_2

* 2022-04-17_3

* 2022-04-17_4

* 2022-04-25

* 2022-05-02_V1

* 2022-05-06_V1

* 2022-05-07_V1

* Update loss.py

* Update loss.py

* Update loss.py

* Update loss.py

* Update loss.py

* Update loss.py

* Update loss.py

* Update loss.py

* Update loss.py

* Update loss.py

* 2022-06-01_pre-commit

* 2022-06-05

* 2022-06-06

* 2022-06-07

* 2022-06-07_V2

* Refine covnert_var_shape in dy2st

* Fix UT error

* Return paddle.shape only when has_negative and fix test_tensor_shape error

* Fix test_jit_save_load error

* support set_grad_ivar for eager

* support set_grad_ivar for eager

* support set_grad_ivar for eager