* add flow.relu

* add test_relu.py

* fix for comments

* add docstring for oneflow.relu

* fix bugs

* add docstring for flow.Tensor.relu

* fix for comments

* fix for comments v2

* auto format by CI

* fix license bug

* auto format by CI

* fix tets_quantization.py bug
Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add reduce_prod op

* add test_prod

* add prod backward

* modify  backward

* mod backward

* mod prod_test

* add note

* mod prod test

* conflict fixed

* conflict fixed

* auto format by CI
Co-authored-by: NYongtaoShi <73167956+YongtaoShi@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* feat(no_grad): support no_grad decorator

* feat(AutogradMode): export flow.autograd_mode

* feat(GradMode): export some grad_mode class

* docs(GradMode): export documents

* refine

* docs(GradMode): export document for is_grad_enabled

* auto format by CI

* fix(GradMode): fix single client bug

* fix bug
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* add randperm with test and docs

* format code

* format

* fix docs

* format the code and add more tests

* format code

* Update test_randperm.py

* Update randperm.py

* add head

* format codes

* docs

* Update test_randperm.py

* Update test_randperm.py

* add more tests

* format

* Update randperm.py

* Update randperm_kernel.cu

* Update randperm_kernel.cu

* Update randperm_kernel.cpp

* reconstruct the code

* format the code

* 2

* s

* 1

* 1

* refine

* add more test

* refine code

* fix according to comment

* add more test
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add flow.rand

* update docstr

* update docstr

* add consistent_rand, add more tests

* update random op

* refine

* refine, add range and int type to uniform_kernel

* refine

* refine

* update doc

* update doc

* Refactor UniformDistribution

* fix
Co-authored-by: Nhjchen2 <chenhoujiangcug@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* feat(Tensor): support Tensor.__bool__()

* test(Tensor): add tensor to bool test

* docs(Tensor): refine is_nonzero document

* format

* fix(Tensor): fix Tensor.__bool___ bug

* auto format by CI

* fix(instancenorm): fix merge bug

* fix(*): fix merge bugs
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Ncheng cheng <472491134@qq.com>

* Remove single_client api files

* Remove Single-Client API in oneflow default python

* revert oneflow.env

* refine docs

* fix watcher and delete useless python callback

* Remove DestroyGlobalWatcher

* Revert test_deconv for CI

* fix test tol
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add narrow user op

* add narrow functor

* add narrow module

* add narrow gradient func

* add split functor and module

* testcase

* fix test bug

* testcase

* add doc and doctest

* fix docs
Co-authored-by: NBBuf <1182563586@qq.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add index_select op

* add CI testCase, update index_select op

* solve some problems of code reviewing

* update CI test

* update index_select, add oundary conditions

* create html,  changed code format

* delete module, change for using function call

* add  annotation

* solve several problems of reviewing code

* add  annotation of param range

* modified  tensor index to a longer tensor
Co-authored-by: NZhenhua <1209435+hengzi@users.noreply.github.com>
Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Dev logical_xor module

* Add single_client api, modify logical_xor implementation, add more test case,

* Update binary_func.h

* Update tensor.rst

* Update logical_xor.py

* Update binary_func.h

* make of_format

* add pytorch auto test

* remove float test

* add test case

* test modified

* delete help.py

* modify name

* make of_format

* modify single-client example result

* remove module not required,call function directly. restore changes under single_client folder

* remove unnecessary nn module package, add shape test of logical_xor

* auto format by CI
Co-authored-by: NZhenhua <1209435+hengzi@users.noreply.github.com>
Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* nonzero-init

* auto format by CI

* modify-docstring
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* add flow.randn

* auto format by CI

* refine

* add mean and std as attr to normal_op/kernel/distribution

* refine

* refine

* keep module, fix docstring to pass CI

* refine
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Delete part of the interface

* auto format by CI

* Delete comment

* Delete comment

* delete oneflow.rst advanced
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* add api

* add api

* add api

* fix comments

* fix complie bug
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: NYao Chi <later@usopp.net>

* fix bugs in shareing EagerBlobObject::blob_desc_.shape and EagerBlobObject::blob_.shape

* feat(EmptyOp): add flow.empty

* docs(EmptyOp): add doctest and refine document

* docs(EmptyOp): refine document

* refactor(Tensor): Tensor constructor use empty_op

* refactor(Tensor): remove useless code

* feat(EmptyOp): support construct in given device and add
               consistent_empty op

* feat(EmptyOp): support unpacked tuple shape

* refine array functor code

* docs(EmptyOp): update empty op document

* refine code

* docs(EmptyOp): add test and document for consistent empty op

* update document

* fix merge bugs

* fix(*): fix infer distribution

* test(EmptyOp): fix ConsistentEmptyOp CPU_ONLY test bug

* fix(*): init shape when InitBlob

* fix(*): Constant and Empty Op use broadcast sbp

* fix(indexing): replace MakeTensor with functional::Empty

* fix(*): fix compile bug

* refine code

* fix(nnGraph): make eager tensor

* auto format by CI

* fix(Stride): infer stride before initializing shape
Co-authored-by: NXinqi Li <lixinqi0703106@163.com>
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* Dev logical_and/or modules

* Add docs

* Add autotest

* Add single_client api
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* startup of dev scatter ops

* use dim scatter base class

* refine(using binop to abstract scatter update and add

* refine (use macros to implement kerenl class and functors)

* refine(description for register scatter ops/kernels)

* refine

* add inplace ops

* python wraper scatter_add inplace

* dev inplace ops

* refine dim_gather (using macros register mechanism)

* add grad of scatter_add_like

* refine (add src, like versions for scatter)

* refine src/like tensor

* gather refine(no need outplace/inplace versions)

* reformat

* refine

* test case of dim scatter

* test case for dim_scatter_add_like

* 1n2d test case for dim_scatter_add_like

* refine scatter sbp

* fail to sccater_add_like on 1n2d

* refing sbp

* refine test case, unify add and update like ops

* test case for scatter_add/update like ops finished

* test cases for scatter ops

* refine, merge test class

* startup of api docs

* add scatter api docs and assertion in python

* fix make error but still segment fault

* annotate sbp infer

* rewrite scatter kernel logic

* remove inplace proposal and fix macro name

* remove outdated atomic add

* move sbp infer

* add const and throw error

* add check

* set grad op

* add scatter scalar

* add scatter scalar gpu kernel

* add torch style backprop

* add torch style backprop check

* align with master

* remove redundant sbp check

* add test

* add float16n register

* fix sbp

* fix sbp

* add api doc

* make format

* add new line

* refine

* revert dim gather

* extract dim_scatter_add

* extracat scatter update ops

* add add/update functor

* rewrting by functors

* refine

* remove dim_gather_scatter_uitl.h

* add blank line

* refine macros for registering kerenls

* refine dim_scatter_scalar files name

* refine

* refine register ops

* refine

* add F.dim_scatter_scalar

* add scatter op

* refine docstr

* add scatter reduce arg

* finally(!): a draft for scatter constitent with pytroch

* change import package name

* remmove lazy test and add scatter_add and scatter_mul

* startup of scatter backward op

* add backward for scatter

* scatter ops backward finished

* add scatter, scatter_add test cases

* remove useless scatter_update_like

* reformat

* refine test cases

* refine according to comments

* revert op_exprt_helper

* fixed index element

* fix scatter update like expr for dim gather backward
Co-authored-by: Ndoombeaker <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add autotest tan tanh floor arctanh

* add autotest tan floor tan

* Add autotest  for log1p

* Code format

* delete no use import

* add eye op alignment

* amend pytorch test

* delete merge error branch

* Delete log1p.py

delete merge error branch

* amend code because the list of master change

* amend eye test code

* amend eye docsting

* amend eye docsting

* amend eye docstring

* autotest test_eye

* auto format by CI

* amend param of eye
Co-authored-by: NZhenhua <huangzhenhua@zhejianglab.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add activation

* rename swish to silu

* add selu

* add four activation op

* add softsign test

* add silu mish selu softsign

* Add softsign docs

* Add functional impl

* small fix for softsign backward

* remove flow.mish test

* add silu module test

* add selu test

* fix docs

* fix softsign docs

* fix format

* fix static cast

* merge master functional api yaml

* add torch style unittest

* Remove assert and add torch unittest

* add tensor def

* remove softsign test temporary

* add return maybe ok

* migrate nn ops to single_client dir

* migrate unittest

* remove lazy unittest

* add unittest

* fix to new directory

* Remove useless docs and single client test

* add doc

* fix docs

* add docs in oneflow and tensor namespace

* add torch autotest

* fix to new autotest

* remove outdated python code

* remove useless docs

* enlarge unittest tolerance

* Add static cast for const value

* skip softsign unittest

* skip tensor softsign
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* change doctest code in abs.py

* del >>> flow.enable_eager_execution() in  modules/abs.py

* del @experimental_api

* del @experimental_api

* slove doctest not pass

* change abs.py for test

* refine acos

* remove enable_eager_execution() in oneflow/python

remove @experimental_api decorator for apis, except that ones
having same name with single client api.

* remove all import oneflow.experimental in doctest

* replace oneflow.experimental with oneflow in docstr

* reformat

* auto format by CI

* remove signle client twice api

* refine

* remove experimental of nn

* okokok

* remove experimental in test/modules and test/tensor

* remove experimental in test/dataloader

* remove experimental call in nn/modules

* remove experimental from nn/ framework/ test_utils

* remove experimental in functionalpy

* remove experimental of new merge

* skip fmod grad test

* remove experimental of fmod

* fix error when do multi case test on graph

* remove outdated rst files

* refine and add new rst files

* format

* revert test_fmod.py

* remove all experimental in python/nn dir (done!)

* remove experimental in test/graph

* add linagl

* refine rst

* refine scatter_nd

* refine oneflow.rst

* refine load_minst's docstring

add image.rst module.rst

refine rst files

* reformat

* remove Graph api

* fix file name

* fix index
Co-authored-by: NYaoChi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Nstrint <xiaoyulink@gmail.com>
Co-authored-by: Ntsai <jackalcooper@gmail.com>

* docstring for relu

* docstring for gelu

* docstring for tanh and sigmoid

* docstring add op

* docstring for function

* docstring for job_set_util.py

* docstring for placement_util.py

* update docstring for placement_util.py

* docstring for check_point.py

* docstring for input_blob_def

* docstring for reduce_sum

* docstring for initializer_util.py

* docstring for array_ops.py

* fix import

* fix reshape

* fix doc fmt

* fix doc fmt

* add basic index

* add file

* add docs

* change upper case of oneflow name

* add docs

* add docs

* add docs

* add docs

* add docs

* add docs

* add MANIFEST.in

* add docs

* reorder index

* add docs

* add docs

* add docs

* fix docs

* add docs

* add docs

* add example

* add example

* add docs

* add docs

* add docs

* integrate make copy py scripts for conveniences

* refine changes

* refine docs

* add docs

* add docs

* add docs

* add docs

* reorder and add add docs

* add docs

* add reference link

* add docs

* add docs

* add docs

* add docs

* fix wording

* rm doc entry of deprecated

* rm function not exposed

* fix wording

* add index for losses

* add docs

* add & fix docs

* add ref

* fix wording

* add docs

* add docs

* add reference

* fix docs

* add docs

* add docs

* reorder

* move docs

* add docs

* add docs

* better wording

* rm duplicated relu if

* Dev docstring tutorial (#2421)

* add tutorials

* Update conf.py

* Update quick_start.md

* Update alexnet.md

* fix cap case

* fix wording

* fix docs

* rm docs

* update manifest

* mv docs to new api

* fix

* updates for refactor

* add doc string

* fix docs index

* build docs in ci

* rm year

* fix markdown docs

* add year

* rm duplicated

* Delete MANIFEST.in

* Update Dockerfile

* Update Dockerfile

* add deps

* Delete build.sh

* build docs in CI

* change dir

* fix path

* fix func name

* rename input  x

* fix docs of ImageRandomFlip

* fix docs of slice v2

* check in auto indexing

* rm index of keras.activations

* rm entry of keras.activations
Co-authored-by: NXIE Xuan <xiexuanx2@gmail.com>
Co-authored-by: NCai Shenghang <caishenghang@gmail.com>
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: NQiaoJing <mir-of@outlook.com>