* feat(autograd.Function): add base class define

* format

* feat(autograd.Function): cache FunctionOpExpr in AutogradFunctionBase
                         and pass autograd.Function name to cpp

* feat(autograd.Function): wrapper PyFunction to FType

* fix(autograd.Function): fix wrapper function capture bug

* feat(autograd.Function): support autograd.Function backward

* feat(autograd.Function): refine apply return value

* fix(autograd.Function): fix autograd.Function name bug

* feat(autograd.Function): refine ctx python api

* feat(*): refine apply interface

* test(autograd.Function): fix ctx interface and add test

* feat(autograd.Function): support mark_non_differentiable

* align ctx.saved_tensors interface

* docs(autograd.Function): export documentation

* refine function names

* refine interface

* use py::args instead of py::object

* refine code

* fix(*): fix `func_name` variable conflict with CHECK_JUST

* feat(autograd.Function): support static call

* docs(autograd.Function): update documentation

* refine code

* add JUST
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Interface primitive::BroadcastElementwiseBinary

* refine
Co-authored-by: Nguo ran <360112263@qq.com>

Signed-off-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Matmul kernels use primitive

* refine

* fix
Co-authored-by: Nguo ran <360112263@qq.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix_naive_eager_boxing_checker

* refine eager_p_to_b_kerne and eager_p_to_s_kernel

* initial commit for adding logical_not operator

* adding logical_not op, debugging Dtype related problems

* finished testing locally, need to add tests

* added tests

* added docs and formatted code

* format file

* format file

* remove python wrapper

* modification based on review

* remove redundant code, format file

* modifications based on reviews

* modifications based on review

* fix duplicate license info

* fix docstring

* fix docstring warning
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix sbp for prelu

* auto format by CI
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* imporve roll speed

* imporve speed of len(dims) > 1 cases
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* dev torch style permute kernel

* Refine

* fix batch permute launch condition

* fix batch permute dispatch logic

* remove redundant header file

* simplified check logic

* use permute primitives in transpose kernels

* fix batch permute logic and avoid mod

* remove redundant templates

* fix grid step

* add grid for loop to avoid the elementnum is too large

* fix bug when hw is not divided by tile size

* refine format

* add a copy kernel as a baseline

* remove annotation

* add copy kernel

* add sync

* use batch permute for profile

* add copy tile baseline

* simplify params for copy kernel

* add slow copy kernel

* use mul to instead mod and remove copy

* use movement size = 4 when h w is modify by 2

* Add temp process for half2

* add half2 specialized kernel

* remove redundant license

* simplified code

* fix format

* fix comment

* fix comment

* use bad for loop condition

* merge half2 in load

* fix bad for loop in batch permute

* refine

* use align storage

* refine

* fix comment

* fix comment

* fix format

* add const and remove redundant header file

* remove register macro

* refine cuda code

* fix guoran comment

* fix format

* fix some details

* remove cuda graph

* fix for 0d tensor
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* c++ standard: bump to 14

* remove cplusplus_14.h & use cxx14

* fix python test

* fix .clang-format
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* support create opt in graph

* add comment
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add roll op

* imporve speed

* improve speed when len(dims) == 1

* move some logic to C++

* fix static analysis error

* refine doc

* add roll doc

* refine codes according to review comments

* remove runcudakernel macro
Co-authored-by: NZZK <42901638+MARD1NO@users.noreply.github.com>

* remove most usage of macros INTRUSIVE_*

* rename most INTRUSVE_XXX macros to REFLECTIVE_XXX

* move intrusive::Base to intrusive/base.h

* 1) remove OFFSET_STRUCT_FIELD; 2) mv test cases of HeadFreeList into head_free_list_test.cpp
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refactor vm preschedule

* TryMoveFromWaitingToReady

* revert flying_instruction_cnt

* revert to single position to call DispatchInstruction

* revert several code

* remove is_xxx_hook_empty
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add commit based pip index.html

* use provision for test

* roll back to release nodes
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Async access blob and copy data between python object, and fix deadlock.

* revert

* fix

* refine code style

* optimize treat single as tuple

* Fix tensor numpy api.

* adapt interface to compatiblility test

* auto format by CI

* refine

* Back up the numpy array when copy data from array to tensor async

* fix pybind blob api

* Make sure array is C-style contiguous.

* decrease ref

* fixup

* Move foreign lock helper base into core/common.

* Release GIL before call SpinWaitUntilTimeout
Co-authored-by: NZhanghuihong <garfield.gzhh@gmail.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine sequence_function.h

* refine nn_functor with sequence_function

* refine activation_functor with sequence_function

* refine generator

* refine

* add thne_if

* refine array_functor with sequence_function

* refine

* refine reduce grad funcs with sequence_function

* remove GET_GENERATOR
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add test file at first

* add tripletMarginLoss py code

* module ok

* add  forward test

* amend test code

* delete import torch

* add autotest ok

* delete numpy test code

* amend docstring

* amend loss.py, delete None

* API transfer to C++

* motify module

* delete cout

* delete cout

* Submit some modified code first

* submit vector_norm functor

* matrix norm

* Refine max/min functor (#6359)

merge to dev_tripletMarginLoss

* replace reducemax and reducemin

* amend code error

* motify code

* delete norm2

* delete print

* delete norm2

* delete print

* motify review code

* add assert to c++

* motify review code

* add else

* motify review problem

* add code

* add test code

* motify code delete dim_check

* delete norm.py code

* delete print

* delete print

* delete pu norm

* delete error code

* motify docsting

* auto format by CI

* delete no use num_dims

* delete import torch lib

* delete CI bug code

* motify clip_grad_norm_ resolve autotest bug

* auto format by CI

* motify loss docstring

* motify norm docstring
Co-authored-by: NZhenhua <1209435+hengzi@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* Fix SimplifyPermutation

* fix

* fix typo

* fix

* add test

* fix init
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* initalizes cuda context in kernel of copy

* debugging

* add call once

* remove redundant code

* changes based on review

* delete redundant code

* fix clang compile error
Co-authored-by: NHoujiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Signed-off-by: Ndaquexian <daquexian566@gmail.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix typo

* use cuda elementwise
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* dev masked fill

* refine

* make static_check happy
Co-authored-by: NHoujiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* interface primitive::softmax

* refine
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Remove primitiveCudaGraphSupport

* fix
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* tensor_slice_copier use copy_nd_primitive

* fix

* 0-dim tensor

* refine

* rm memory_copier
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* restruct part1

* restruct stack

* refine

* add meshgrid functor

* restruct squeeze

* refine

* refine

* refine

* refine

* refine

* refine

* refine

* auto format by CI

* refine
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Apply2

* fix

* CastFunctor::Apply2
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>