Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* FusedScaleTrilFunctor

* grad func

* add test

* fix bugs

* rm comment

* fix test

* fix conflict
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add test consistent tensor file

* add consistensor grad  test

* support  split(0)

* Separate consistent tensor test

* fix  line

* fix  errror

* Fix duplicate code

* Delete fill_

* Add todo

* Update python/oneflow/test/tensor/test_consistent_tensor.py
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>

* merge master

* fix  merge master  error

* master has fixed this problem

* fix  merge master error

* auto format by CI

* Master fixed this bug

* add consistent tensor grad
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add log1p autotest

* add log autotest

* add autotest fake_code_gen

* refine code

* add code color

* add clear_note_fake_program

* code format

* fix clear list bug

* fix clear list bug

* code format

* delete useless file

* refine tensor method test

* fix comments
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* rename OpExprInterpState to AutoGradCaptureState

* fix

* fix

* delete comment
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Fix backward bug in 3d

* add conv3d

* remove params in 1d

* fix format

* fix docs
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: NTianyu Zhao <guikarist@gmail.com>

* maybe: add JUST_MSG and CHECK_JUST_MSG

* maybe: add test

* maybe: support ostream && fix test

* maybe: put to private_details

* maybe: fix name

* error: clang format
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix bug scalar

* add test
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refactor static_check

* fix bug in IsOutArg
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* CI: add clang-tidy check to test.yaml

* CI: fix checkout

* CI: fix diff

* CI: add ccache

* CI: fix deps

* CI: fix typo

* CI: fix typo

* CI: try to remove ccache for c

* CI: fix opencv ccache

solve OneFlow-Inc/oneflow#5889

* wrap flow.nn.init.* with flow.no_grad()
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* auto format by CI
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix_ddp_bug_on_8_process

* fix_ddp_bug_on_8_process

* refactor one_hot api

* fix test

* num_classes default to -1
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix missing error info in XXError

* refine

* Correct spelling mistake

* refine

* fix bug

* refine

* Use protobuf's reflection to get error msg from error type

* refine

* fix bug
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* print consistent tensor data

* refine

* refine

* refine

* refine

* fix bug: consistent tensor's cur_rank_phy_tensor device is gpu

* refine

* change comment

* refine

* return ... when sbp == flow.sbp.split(x), x!=0

* fix bug

* refine

* Update python/oneflow/framework/tensor_str.py
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>

* refine

* revert device tag to device type change

* add op test

* auto format by CI

* format
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* single device
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* support consistent load and save
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* add consistent init test
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* reformat
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* refine
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* refine
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* refine CpuBroadcast(py::bytes* in, int64_t root)
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* refine CpuBroadcast(py::bytes* in, int64_t root)
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* refine CpuBroadcast(py::bytes* in, int64_t root)
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* auto format by CI

* fix test script path
Signed-off-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* add clip_grad to optimizer

* add clip grad unit test to sgd optimizer

* add sgd graph clip grad test

* refine

* format

* refine

* fix

* merge master

* refine

* refine
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* where broadcast forward

* delete py broadcast code

* delete op_expr

* add where backward broadcast gradient

* scalar x op

* add where scalar_x, scalar_y, scalar_xy ops & kernels

* add where scalar_x, scalar_y, scalar_xy functors

* fix bug

* add where backward

* add pybind api

* add testcase

* fix

* fix
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Remove GlobalWorkStreamId/GlobalThrdId

* refine
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: NHoujiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add randperm with test and docs

* format code

* format

* fix docs

* format the code and add more tests

* format code

* Update test_randperm.py

* Update randperm.py

* add head

* format codes

* docs

* Update test_randperm.py

* Update test_randperm.py

* add more tests

* format

* Update randperm.py

* Update randperm_kernel.cu

* Update randperm_kernel.cu

* Update randperm_kernel.cpp

* reconstruct the code

* format the code

* 2

* s

* 1

* 1

* refine

* add more test

* refine code

* fix according to comment

* add more test
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* lrs

* prototype lrs

* add cosine lr scheduler conf

* add opt group

* test opt group

* add comment

* add comment

* add return for lr conf

* check same optimizer in the opt group

* check same optimizer in the opt group

* check cosine lr compute

* cosine lr_sch pass test

* add_optimizer

* rm outdated comment

* make lr_sheduler optional and add test
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* ref count del of session and env

* auto format by CI

* add test

* pass graph in global test

* auto format by CI

* rm cuda

* rm debug lof

* rm debug info

* move graph del sync from c to python

* rm log

* auto format by CI

* refine test to pass ci

* fix env sync call

* auto format by CI

* deal with graph destruction when graph not compiled

* deal with not compiled graph destruction

* print log

* deal with single client sync

* auto format by CI

* rm py object ref

* rm python obj ref count

* add log of env

* address comment

* auto format by CI
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix(*): fix inplace op circle reference bug

* refine code

* format
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* process api dtype

* process api tensor

* small fix

* fix to name func

* still error

* still error

* revert master

* fix symbol dtype part1

* import error

* still error

* still error

* fix symbol

* add Dtype4Datatype function

* work in process

* undefined symbol of tensorbuffer

* macro bug

* construction func return symbol dtype

* process deprecated api

* fix name

* fix bug

* change to use reference

* fix format

* fix name

* small fix

* fix symbol dtype

* modify functor datatype to symbol dtype

* fix to symbol dtype

* add object as for symbol dtype

* modify return symbol dtype

* modify functor

* modify autograd section

* modify autograd

* modify core

* modify back for functional

* modify functional impl

* modify value types

* modify value types

* fix args

* modify tensor

* modify back

* fix comment

* small fix

* auto format by CI

* fix tensor buffer bug

* modify new interface

* modify new interface

* remove annotation

* auto format by CI

* need refine

* remove dtype4datatype

* modify use Get

* auto format by CI

* small fix

* auto format by CI

* remove annotation

* houjiang fix to symbol dtype pointer

* fix dtype

* fix format

* remove annotation

* small fix

* add null ptr support

* auto format by CI

* use symbol dtype

* fix

* auto format by CI

* small fix

* fix in lazy op interpreter

* auto format by CI

* fix to use symbol dtype

* small fix to use equal instead of is in single client

* fix conflict

* Format

* Fix to use symbol dtype
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Ncheng cheng <472491134@qq.com>