* Fix treat_args_as_inlist logic

* Align pytorch interface

* auto format by CI

* Fix empty and reformat

* fix

* auto format by CI

* fix merge
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add randint

* add

* add doc test

* Update randint_kernel.cu

* Update randint_kernel.cpp

* Update randint_op.cpp

* reconstrcut

* refine the code

* add test

* add test

* add test

* format

* Dev randint refine (#5981)

* disable backward pass consistent tensor meta check. (#5871)

* disable backward pass consistent tensor meta check.

* auto format by CI
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* ddp broadcast params and buffers (#5913)

* ddp broadcast params and buffers
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* auto format by CI
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* add clang tidy target (#5957)

* add clang tidy target

* fix a bug

* refine

* refine

* reformat
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* cfg: add move assignment operator for performance (#5962)
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add zhangshen op-test (#5600)

* add some op-test

* fix dims_error in my branch

* Fix the bad backward kernel function by using 'cuda::atomic::Add' (#5614)

* Test `nn.AdaptiveAvgPoolXd` (#5615)

* Fix the bad backward kernel function by using 'cuda::atomic::Add'

* Support the 'NoneType' annotation

* Support objects of 'collections.abc.Iterable' as 'output_size'

* Test with all cases of 'output_size'

* Update adaptive_pool_gpu_kernel.cu

* Skip testing `nn.AdaptiveAvgPool3d` for the current PyTorch

* remove some useless test

* Format TODO

* Add the assertion messages for 'output_size'

* Reformat codes

* Remove raw tests for `flow.negative`

* Remove unnecessary codes and add the assertion messages

* Merge updates for 'generators.py' from master

* Remove unnecessary 'random()'

* Delete the separate test for `AvgPool2d`

* Fix import paths

* Fix import problems

* Remove the PyTorch import

* Denote the annotations for `tile` and `repeat` ops

* Add the test for `nn.AvgPool1d`

* Choose better generators for `nn.MaxPoolXd`

* Randomly choose `dilation` and default values

* auto format by CI

* Test more kwargs for `nn.AvgPoolXd`

* Add tests for `return_indices`

* auto format by CI
Co-authored-by: NTianyu Zhao <guikarist@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* fix wrong names (#5951)

* fix wrong names

* auto format by CI

* refine

* auto format by CI
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Enable more checkers for clang-tidy in CI (#5738)

* CI: enable more checkers for clang-tidy

* .clang-tidy: remove cppcoreguidelines-pro-type-vararg

* CI: remove duplicate checkers

* CI: remove clang-analyzer-alpha.deadcode.*

* .clang-tidy: add performance-*

* oneflow/core/eager: remove unnecessary malloc & free

* .clang-tidy: add clang-analyzer-cplusplus.* to werror

* user_kernel: remove useless move

* quantization_aware_training: fix move return

* .clang-tidy: add google-*

* CI: fix clang tidy command

* CI: fix test
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Feat grad mode classes (#5956)

* feat(no_grad): support no_grad decorator

* feat(AutogradMode): export flow.autograd_mode

* feat(GradMode): export some grad_mode class

* docs(GradMode): export documents

* refine

* docs(GradMode): export document for is_grad_enabled

* auto format by CI

* fix(GradMode): fix single client bug

* fix bug
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* extract_consistent_to_consistent_op_expr (#5870)

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* refactor functional::ToConsistent

* refactor GetNdSbp

* Update eager_consistent_op_interpreter.cpp

* Update eager_mirrored_op_interpreter.cpp

* fix error

* fix error

* auto format by CI

* Update nd_sbp.h

* refine identity boxing

* fix sync checkmeta error

* avoid consistent id check in lazy
Co-authored-by: NXinqi Li <lixinqi0703106@163.com>
Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache (#5970)

* add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache

* skip test targets of re2
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check: fix clang-tidy-diff commands (#5972)

* check: fix clang-tidy-diff commands

* CI: fix step names
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cpu mpi (#5865)

* cuda base cpu mpi boxing

* cpu_mpi

* fix conflicts

* add cpu mpi unittests

* more checks and unittests

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* more test_consistent_cast unittests

* refactor functional::ToConsistent

* refactor GetNdSbp

* fix compiler complaints

* refactor GetDevice4CurrentProcessCtx

* fix error
Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix_bug_test_tensor_str (#5958)

* fix bug int test_tensor_str

* format

* fix comment

* fix bug to(cuda) is unavailable in cpu env
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* common/error: fix build error in mac (#5971)
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Prevent running oneflow in forked subprocess (#5976)

* prevent_running_oneflow_in_forked_subprocess

* add line change

* IsFork => IsForkedSubProcess

* auto format by CI
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* refine randint
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: NPeihong Liu <mosout@qq.com>
Co-authored-by: NTwice <i@twice.moe>
Co-authored-by: NZhangShen <55383772+zhangshen12356@users.noreply.github.com>
Co-authored-by: NTianyu Zhao <guikarist@gmail.com>
Co-authored-by: NLuyang <flowingsun007@163.com>
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>
Co-authored-by: NXinqi Li <lixinqi0703106@163.com>
Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Nliufengwei0103 <2472937968@qq.com>

* refine

* refine

* auto format by CI

* refine

* Update functional_api.yaml

* Update functional_api.yaml

* refine the code

* auto format by CI

* refine

* fix ci error

* fix test

* auto format by CI

* fixtest

* refine code

* auto format by CI

* refine code

* auto format by CI

* fix ci fail

* remove redefination api

* fix ci test

* auto format by CI

* fix consistency with torch

* auto format by CI

* unittest fixed

* fix doctest
Co-authored-by: NBowen Chen <bob2420083992@gmail.com>
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: NPeihong Liu <mosout@qq.com>
Co-authored-by: NTwice <i@twice.moe>
Co-authored-by: NZhangShen <55383772+zhangshen12356@users.noreply.github.com>
Co-authored-by: NTianyu Zhao <guikarist@gmail.com>
Co-authored-by: NLuyang <flowingsun007@163.com>
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>
Co-authored-by: NXinqi Li <lixinqi0703106@163.com>
Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Nliufengwei0103 <2472937968@qq.com>

* add randperm with test and docs

* format code

* format

* fix docs

* format the code and add more tests

* format code

* Update test_randperm.py

* Update randperm.py

* add head

* format codes

* docs

* Update test_randperm.py

* Update test_randperm.py

* add more tests

* format

* Update randperm.py

* Update randperm_kernel.cu

* Update randperm_kernel.cu

* Update randperm_kernel.cpp

* reconstruct the code

* format the code

* 2

* s

* 1

* 1

* refine

* add more test

* refine code

* fix according to comment

* add more test
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* process api dtype

* process api tensor

* small fix

* fix to name func

* still error

* still error

* revert master

* fix symbol dtype part1

* import error

* still error

* still error

* fix symbol

* add Dtype4Datatype function

* work in process

* undefined symbol of tensorbuffer

* macro bug

* construction func return symbol dtype

* process deprecated api

* fix name

* fix bug

* change to use reference

* fix format

* fix name

* small fix

* fix symbol dtype

* modify functor datatype to symbol dtype

* fix to symbol dtype

* add object as for symbol dtype

* modify return symbol dtype

* modify functor

* modify autograd section

* modify autograd

* modify core

* modify back for functional

* modify functional impl

* modify value types

* modify value types

* fix args

* modify tensor

* modify back

* fix comment

* small fix

* auto format by CI

* fix tensor buffer bug

* modify new interface

* modify new interface

* remove annotation

* auto format by CI

* need refine

* remove dtype4datatype

* modify use Get

* auto format by CI

* small fix

* auto format by CI

* remove annotation

* houjiang fix to symbol dtype pointer

* fix dtype

* fix format

* remove annotation

* small fix

* add null ptr support

* auto format by CI

* use symbol dtype

* fix

* auto format by CI

* small fix

* fix in lazy op interpreter

* auto format by CI

* fix to use symbol dtype

* small fix to use equal instead of is in single client

* fix conflict

* Format

* Fix to use symbol dtype
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Ncheng cheng <472491134@qq.com>

* add flow.rand

* update docstr

* update docstr

* add consistent_rand, add more tests

* update random op

* refine

* refine, add range and int type to uniform_kernel

* refine

* refine

* update doc

* update doc

* Refactor UniformDistribution

* fix
Co-authored-by: Nhjchen2 <chenhoujiangcug@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Rename `parallel_distribution` to `nd_sbp`

* Rename filenames containing `parallel_distribution`

* auto format by CI

* Rename `parallel_distribution` to `nd_sbp`

* auto format by CI

* Rename `parallel_distribution` to `nd_sbp`

* auto format by CI
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add flow.randn

* auto format by CI

* refine

* add mean and std as attr to normal_op/kernel/distribution

* refine

* refine

* keep module, fix docstring to pass CI

* refine
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* rebase

* check in gen py

* merge master and fix bugs

* address pr comments

* address pr comments

* auto format by CI

* functional python_arg

* auto format by CI

* remove unused files

* fix return type error on gcc 4.8.5
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* auto format by CI

* fix return type error in xrt
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* fix tick ibn sbp signature

* auto format by CI
Co-authored-by: Ntsai <jackalcooper@gmail.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

* add bernoulli module

* fix doc test

* add bernoulli functor

* make changes according to review

* refine (#5415)

* fix
Co-authored-by: NBowen Chen <bob2420083992@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>