* support create opt in graph

* add comment
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add roll op

* imporve speed

* improve speed when len(dims) == 1

* move some logic to C++

* fix static analysis error

* refine doc

* add roll doc

* refine codes according to review comments

* remove runcudakernel macro
Co-authored-by: NZZK <42901638+MARD1NO@users.noreply.github.com>

* remove most usage of macros INTRUSIVE_*

* rename most INTRUSVE_XXX macros to REFLECTIVE_XXX

* move intrusive::Base to intrusive/base.h

* 1) remove OFFSET_STRUCT_FIELD; 2) mv test cases of HeadFreeList into head_free_list_test.cpp
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refactor vm preschedule

* TryMoveFromWaitingToReady

* revert flying_instruction_cnt

* revert to single position to call DispatchInstruction

* revert several code

* remove is_xxx_hook_empty
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add commit based pip index.html

* use provision for test

* roll back to release nodes
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Async access blob and copy data between python object, and fix deadlock.

* revert

* fix

* refine code style

* optimize treat single as tuple

* Fix tensor numpy api.

* adapt interface to compatiblility test

* auto format by CI

* refine

* Back up the numpy array when copy data from array to tensor async

* fix pybind blob api

* Make sure array is C-style contiguous.

* decrease ref

* fixup

* Move foreign lock helper base into core/common.

* Release GIL before call SpinWaitUntilTimeout
Co-authored-by: NZhanghuihong <garfield.gzhh@gmail.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine sequence_function.h

* refine nn_functor with sequence_function

* refine activation_functor with sequence_function

* refine generator

* refine

* add thne_if

* refine array_functor with sequence_function

* refine

* refine reduce grad funcs with sequence_function

* remove GET_GENERATOR
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add test file at first

* add tripletMarginLoss py code

* module ok

* add  forward test

* amend test code

* delete import torch

* add autotest ok

* delete numpy test code

* amend docstring

* amend loss.py, delete None

* API transfer to C++

* motify module

* delete cout

* delete cout

* Submit some modified code first

* submit vector_norm functor

* matrix norm

* Refine max/min functor (#6359)

merge to dev_tripletMarginLoss

* replace reducemax and reducemin

* amend code error

* motify code

* delete norm2

* delete print

* delete norm2

* delete print

* motify review code

* add assert to c++

* motify review code

* add else

* motify review problem

* add code

* add test code

* motify code delete dim_check

* delete norm.py code

* delete print

* delete print

* delete pu norm

* delete error code

* motify docsting

* auto format by CI

* delete no use num_dims

* delete import torch lib

* delete CI bug code

* motify clip_grad_norm_ resolve autotest bug

* auto format by CI

* motify loss docstring

* motify norm docstring
Co-authored-by: NZhenhua <1209435+hengzi@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* Fix SimplifyPermutation

* fix

* fix typo

* fix

* add test

* fix init
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* initalizes cuda context in kernel of copy

* debugging

* add call once

* remove redundant code

* changes based on review

* delete redundant code

* fix clang compile error
Co-authored-by: NHoujiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Signed-off-by: Ndaquexian <daquexian566@gmail.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix typo

* use cuda elementwise
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* dev masked fill

* refine

* make static_check happy
Co-authored-by: NHoujiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* interface primitive::softmax

* refine
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Remove primitiveCudaGraphSupport

* fix
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* tensor_slice_copier use copy_nd_primitive

* fix

* 0-dim tensor

* refine

* rm memory_copier
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* restruct part1

* restruct stack

* refine

* add meshgrid functor

* restruct squeeze

* refine

* refine

* refine

* refine

* refine

* refine

* refine

* auto format by CI

* refine
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Apply2

* fix

* CastFunctor::Apply2
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Matmul Primitive

* refine

* fix cublasComputeType_t
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Suppport sparse_softmax_cross_entropy_with_logits

* add lazy mode test

* Modify after review

* Add 2d sbp code

* Support 2D eager consistent

* remove log

* remove to_consistent in gradident

* remove useless change

* remove useless log

* auto format by CI

* Unittest use pytorch

* update follow review

* fix static check

* fix
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add Nightly Docker Image Status badge in readme

* refein

* address review
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Support 1d targets as input

* fix bugs

* reference type

* suppress strict-overflow error
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* fix single client

* fix grad check
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix bug in AddInputOutputOpsPass: check existence of key in HashMap(inferface_lbi2scope_sym_id)

* fix bug in add_input_output_ops_pass.cpp: use iterator to avoid search twice

* fix bug when tensor access _tensor_buffer_shapes_and_dtypes

* remove unnecessary header

* format

* throw RuntimeError & add test

* Update oneflow/api/python/utils/tensor_utils.cpp
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

* celu op init

* register grad

* add inplace support

* add test & document

* clang-format

* add doc & inplace test case

* celu functional set alpha default value

* set celu_grad alpha default value

* inv_alpha & alpha

* remove type cast

* add license

* use clang-format-9 git status! dont't use clang-format-6

* set intial value for CeluCaptureState
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

* optional: refactor value_or to allow auto deref & prevent ref dangling

* use value_or

* use value_or

* fix compiler error: invalid abstract return type
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* style(*): fix typo

* style(*): meta_datas_ -> meta_data_
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>