* PlacedNdSbp

* 1) fix compiler complaints; 2) add an unittest

* tensor.cpu/tensor.cuda

* register tensor dtype modules

* export api docs

* fix comments

* refine

* register module methods

* refine

* format

* add testcases

* upper rmsprop tol

* refine

* add more test case

* auto format by CI

* skip test if only cpu

* auto format by CI

* refine

* refine

* fix test

* auto format by CI
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refactor to

* test

* call cast when device_type is same

* add fused_self_attention functor

* add fused_self_attention gradients

* add test case

* fix backward immpl bug

* fix bug

* fix bug

* fix backward bug

* fix comments

* code format

* fix comment

* fix ci error

* fix ci error

* add init FusedSelfAttentionInterpState struct
Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* dos2unix
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* remove automerge on fail
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* test ci
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* Revert "test ci"

This reverts commit 0f1051f3.

Signed-off-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add sys conf

* format

* refine warmup api

* refine config

* format

* add WarmupLR

* add warmup lr

* refactor graph folder structure

* export graph

* add amp conf

* deal with lr init step and test cpu

* add WarmUpLR

* add flow config

* GradScaler and config

* rm useless code

* rm useless code

* add api_enable_cudnn_fused_normalization_add_relu

* move config

* move amp

* refactor WarmUpLrScheduler

* add comment

* add arg
Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* reemove redundant codes in ModuleDict

* auto format by CI

* add moduledict test

* refine

* refactor test case

* auto format by CI

* auto format by CI

* refine

* refine

* refine  docs format
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add flow.relu

* add test_relu.py

* fix for comments

* add docstring for oneflow.relu

* fix bugs

* add docstring for flow.Tensor.relu

* fix for comments

* fix for comments v2

* auto format by CI

* fix license bug

* auto format by CI

* fix tets_quantization.py bug
Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Signed-off-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix var impl bug

* add more test_case

* refine

* add tensor method test

* fix comment

* auto format by CI

* fix comments

* fix ci error

* fix run ci error
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add reduce_prod op

* add test_prod

* add prod backward

* modify  backward

* mod backward

* mod prod_test

* add note

* mod prod test

* conflict fixed

* conflict fixed

* auto format by CI
Co-authored-by: NYongtaoShi <73167956+YongtaoShi@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* use clang-tidy-diff.py return code and add -warnings-as-errors
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* fix python2 shebang
Signed-off-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* prevent_running_oneflow_in_forked_subprocess

* add line change

* IsFork => IsForkedSubProcess

* auto format by CI
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix bug int test_tensor_str

* format

* fix comment

* fix bug to(cuda) is unavailable in cpu env
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* cuda base cpu mpi boxing

* cpu_mpi

* fix conflicts

* add cpu mpi unittests

* more checks and unittests

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* more test_consistent_cast unittests

* refactor functional::ToConsistent

* refactor GetNdSbp

* fix compiler complaints

* refactor GetDevice4CurrentProcessCtx

* fix error
Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check: fix clang-tidy-diff commands

* CI: fix step names
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache

* skip test targets of re2
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* refactor functional::ToConsistent

* refactor GetNdSbp

* Update eager_consistent_op_interpreter.cpp

* Update eager_mirrored_op_interpreter.cpp

* fix error

* fix error

* auto format by CI

* Update nd_sbp.h

* refine identity boxing

* fix sync checkmeta error

* avoid consistent id check in lazy
Co-authored-by: NXinqi Li <lixinqi0703106@163.com>
Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* feat(no_grad): support no_grad decorator

* feat(AutogradMode): export flow.autograd_mode

* feat(GradMode): export some grad_mode class

* docs(GradMode): export documents

* refine

* docs(GradMode): export document for is_grad_enabled

* auto format by CI

* fix(GradMode): fix single client bug

* fix bug
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* CI: enable more checkers for clang-tidy

* .clang-tidy: remove cppcoreguidelines-pro-type-vararg

* CI: remove duplicate checkers

* CI: remove clang-analyzer-alpha.deadcode.*

* .clang-tidy: add performance-*

* oneflow/core/eager: remove unnecessary malloc & free

* .clang-tidy: add clang-analyzer-cplusplus.* to werror

* user_kernel: remove useless move

* quantization_aware_training: fix move return

* .clang-tidy: add google-*

* CI: fix clang tidy command

* CI: fix test
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix wrong names

* auto format by CI

* refine

* auto format by CI
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add some op-test

* fix dims_error in my branch

* Fix the bad backward kernel function by using 'cuda::atomic::Add' (#5614)

* Test `nn.AdaptiveAvgPoolXd` (#5615)

* Fix the bad backward kernel function by using 'cuda::atomic::Add'

* Support the 'NoneType' annotation

* Support objects of 'collections.abc.Iterable' as 'output_size'

* Test with all cases of 'output_size'

* Update adaptive_pool_gpu_kernel.cu

* Skip testing `nn.AdaptiveAvgPool3d` for the current PyTorch

* remove some useless test

* Format TODO

* Add the assertion messages for 'output_size'

* Reformat codes

* Remove raw tests for `flow.negative`

* Remove unnecessary codes and add the assertion messages

* Merge updates for 'generators.py' from master

* Remove unnecessary 'random()'

* Delete the separate test for `AvgPool2d`

* Fix import paths

* Fix import problems

* Remove the PyTorch import

* Denote the annotations for `tile` and `repeat` ops

* Add the test for `nn.AvgPool1d`

* Choose better generators for `nn.MaxPoolXd`

* Randomly choose `dilation` and default values

* auto format by CI

* Test more kwargs for `nn.AvgPoolXd`

* Add tests for `return_indices`

* auto format by CI
Co-authored-by: NTianyu Zhao <guikarist@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add clang tidy target

* fix a bug

* refine

* refine

* reformat
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* ddp broadcast params and buffers
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* auto format by CI
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* disable backward pass consistent tensor meta check.

* auto format by CI
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* GetLocalDepObjectPool

* fix compiler complaints
Co-authored-by: NHoujiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* FusedScaleTrilFunctor

* grad func

* add test

* fix bugs

* rm comment

* fix test

* fix conflict
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add test consistent tensor file

* add consistensor grad  test

* support  split(0)

* Separate consistent tensor test

* fix  line

* fix  errror

* Fix duplicate code

* Delete fill_

* Add todo

* Update python/oneflow/test/tensor/test_consistent_tensor.py
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>

* merge master

* fix  merge master  error

* master has fixed this problem

* fix  merge master error

* auto format by CI

* Master fixed this bug

* add consistent tensor grad
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add log1p autotest

* add log autotest

* add autotest fake_code_gen

* refine code

* add code color

* add clear_note_fake_program

* code format

* fix clear list bug

* fix clear list bug

* code format

* delete useless file

* refine tensor method test

* fix comments
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* rename OpExprInterpState to AutoGradCaptureState

* fix

* fix

* delete comment
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Fix backward bug in 3d

* add conv3d

* remove params in 1d

* fix format

* fix docs
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: NTianyu Zhao <guikarist@gmail.com>

* maybe: add JUST_MSG and CHECK_JUST_MSG

* maybe: add test

* maybe: support ostream && fix test

* maybe: put to private_details

* maybe: fix name

* error: clang format
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>