* add test_cumprod_op

* Revert "add test_cumprod_op"

This reverts commit c96cf6dff5d09ae7d8cc72c1e8ae4369a153aa19.

* recommit

* add error message

* test input(x) initialize

* test use cpu

* update test code

* add test type

* add test case

* solve ci problem

* add complex case test

* add complex case test

* fix review problem

* fix conflict

* fix some docs

* change test case

* change test case

* fix review problems again

* fix docs

* fix inclusivescan bug

This PR supports gradient clip (ClipGradByGlobalNorm) when training with AMP(auto mixed precision).

* add as_extra to dropout op and lrn op

* refine details

* fix dropout op maker

* fix extra op for expand, expand_as, tile, unstack

* Update expand_v2_op.cc

* fix bn/in/squeeze/syncbn extra

* update bn

* update

* update

* add opdef extra

* add reduce mean

* update style

* fix scatter gather bug:

* fix windows ci

* add asExtra for nce op

* fix unittest error in macos

* remove asExtra for is_test

* mark extra attr for unsqueeze2

* debug for inference

* init matrix_rank op, add matrix_rank CPU code and test

* add GPU kernel, remove svd_eigen.h

* add CPU kernel when tol is tensor

* add cpu and gpu code when tol is tensor

* fix CI-ROCM error

* add matrix_rank API describe, fix PR-CI-Py3 error

* fix PR-CI-Windows error, add matrix_rank API test

* delete useless comments

* fix review

* add my code in svd_helper.h

* update doc commets

* remove spaces

multiply supports bool  

* mark WhileOp AsExtra attribute

* revert kX and kOutputs

* add clip_by_norm fp16 kernel

* add ut

* fix scatter_add_nd and gather bug

* fix gather compile error

The bug is that access to mean and var is incorrect, and the array will be out of bounds: the shape of mean and var is [batch_size], and the range of thread idx is 0~feature_size, so mean[idx] and var[idx] is incorrect.

When batch_size=1, the correct access is mean[0] and var[0], and a unit test with batch_size=1 is added.

* merge CMakeList.txt manual

* add platform for changethreadnum

* repair some bugs according to make error

* do nothing just flush CI

* forget change thread num

* add inplace_atol param for check_output_with_place

* Windows

* std:min and std::max should be change because of windows

* add the matmul v2 grad kernel

* relief the test case time

* update the test case for the matmul double grad

* remove the unsed code for the matmul double grad

* update the test case for the double grad matmul

* remove the unused code in dot

* add conv op check for illegal input or attributes

* [NPU] update batch norm op, test=develop

* add NHWC support for bn, test=develop

* [NPU] fix for test_norm_op_npu

* [NPU] add norm_grad

* [NPU] add CheckAxis for axis

* [NPU] delete debug codes

* norm can not use L2Normalize, norm_grad can use L2NormalizeGrad

* [NPU] delete useless codes

* [NPU] optimize norm_grad OpMaker

* Update python import path

* [NPU] log_softmax_grad, test=develop

* remove debug files, test=develop

* update lookup_table_v2 for CANN 5.0.x, test=develop

* - refactoring progressing

- Fix

- compilation fix

- another compilation fix

- refactoring

* - fix

* - compilation fix

* - compilation fix

* - missing set_format

* - compilation fix

* - reverted setting memeory format

* - Brought back format

* - Fix

* - fixes after review

* CI rerun

* CI rerun

* fix for reshape2

* added reviewers sugestions