* Refactor oneflow.Size

* refine

* add pybind11 caster

* Support Shape cast

* refine

* fix size index

* include size header if need export C++ Shape to Python.

* Returns py::tuple instead of tensor tuple, and refine split functional api.

* fix and refine code

* fix compile

* modified split_sizes
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: luqiang-guo <702572275@qq.com>

* add todo

* refine

* add attr

* refine

* refine

* add todo

* refine

* add alias c1 for check-oneflow

* fix

* update scripts

* refine

* fix single client env reinit

* add attr

* save and pass mlir module

* fix

* restore module in kernel

* lower in kernel

* refien

* add scf to std

* update lit

* fmt

* add all passes

* add alisas

* refein

* refein

* add check

* fix pass order

* add TODO

* refein

* create jit exe

* refein

* fix arity

* add check and rpint err

* refein

* refein

* refein

* refein

* refein

* refein

* emiit c

* working

* revert

* add err print

* e2e works

* refein

* refein

* refein

* use STATIC_SWITCH_FUNC

* add log

* rename

* use invoke packed

* refein

* add todo

* refein

* rm log

* fix

* refein

* rm

* refein

* add scf to gpu

* add cmake flag for cuda runner

* add CMAKE_CUDA_COMPILER

* refine

* refien

* register gpu kernel

* refein

* add gpu passes

* refein

* add

* refine

* add ptx to cubin pass

* produce cubin

* add gpu to llvm pass

* refein

* add log

* refien

* link mlir cuda runtime lib

* add note

* make gpu runner available in file check

* rm unused

* add to prevent break

* fix with cuda

* edit mlir by hand to have it run on cuda

* rm useless

* add todo

* upgrade llvm

* refein m,irror scripts

* fix for llvm upgrade

* refein cmake

* fix

* fix for llvm upgrade

* remove unused headers

* refeine

* refein

* refactor

* add

* refine

* refine

* cmake first class cuda support

* refine

* refine

* refein

* refine

* refine

* refine

* refein

* add todo

* refine

* pass shared lib path from py

* prevent redef ONEFLOW_CMAKE_BUILD_TYPE

* refine msg

* fix fmt

* fix fmt

* fix fmt

* refine

* refueb

* fix

* refactor jit function outline

* refein

* rm debug log

* rm unnecessary erase

* use 75

* refein

* add allowFoldingUnitDimReshapes

* refine

* Outline JIT func (#6542)

* check in pass impl

* add test

* check in changes

* add todo

* extract func to create attrs

* refine

* refine and mv bert

* refein LLVM_EXTERNAL_LIT

* refine log user_op::AttrValueUtil::ToCppAttrValue

* fix for nd_sbp

* refine log

* fix warnings

* fix

* leverage input_order and output_order

* save lbn_segment_keys as input output order

* refine

* refein

* add CUDATOOLKIT_BIN_ROOT

* finish todo

* finish todo

* finish todo

* add matmul

* rm repetitive code

* add log

* add unary

* add gather

* refine and add gelu

* fix loc

* add mlir conv op (#6559)

* add mlir conv op

* fix conv2d tabelgen bug

* fix merge compile error

* fix comments

* Update mlir-cuda-75.cmake

* add mlir resnet50 test

* add SI32ArrayAttr
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>

* backport refactoring of translation

* Add resnet50 mlir dialect part ops (#6607)

* add scalar math ops tablegen

* add pool ops

* add bias_add op

* fix comment

* fix comment

* code format

* add reshape op

* add reduce ops and restruct scalar math ops

* fix bug

* fix typo

* address review

* address review

* rm loggin

* address review

* rm logging

* backport variable rename

* add flag ONEFLOW_MLIR_ENABLE_FUSERS
Co-authored-by: NXiaoyu Zhang <35585791+BBuf@users.noreply.github.com>

* feat(autograd.Function): add base class define

* format

* feat(autograd.Function): cache FunctionOpExpr in AutogradFunctionBase
                         and pass autograd.Function name to cpp

* feat(autograd.Function): wrapper PyFunction to FType

* fix(autograd.Function): fix wrapper function capture bug

* feat(autograd.Function): support autograd.Function backward

* feat(autograd.Function): refine apply return value

* fix(autograd.Function): fix autograd.Function name bug

* feat(autograd.Function): refine ctx python api

* feat(*): refine apply interface

* test(autograd.Function): fix ctx interface and add test

* feat(autograd.Function): support mark_non_differentiable

* align ctx.saved_tensors interface

* docs(autograd.Function): export documentation

* refine function names

* refine interface

* use py::args instead of py::object

* refine code

* fix(*): fix `func_name` variable conflict with CHECK_JUST

* feat(autograd.Function): support static call

* docs(autograd.Function): update documentation

* refine code

* add JUST
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* c++ standard: bump to 14

* remove cplusplus_14.h & use cxx14

* fix python test

* fix .clang-format
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Async access blob and copy data between python object, and fix deadlock.

* revert

* fix

* refine code style

* optimize treat single as tuple

* Fix tensor numpy api.

* adapt interface to compatiblility test

* auto format by CI

* refine

* Back up the numpy array when copy data from array to tensor async

* fix pybind blob api

* Make sure array is C-style contiguous.

* decrease ref

* fixup

* Move foreign lock helper base into core/common.

* Release GIL before call SpinWaitUntilTimeout
Co-authored-by: NZhanghuihong <garfield.gzhh@gmail.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix_bug_about_AsyncTransportCtx

* refine CheckVecEqual

* refine

* refine

* minor fix

* Update tensor_utils.cpp

* Update tensor_utils.cpp

* auto format by CI
Co-authored-by: NYu OuYang <xuanjiuye@gmail.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: NYu OuYang <xuanjiuye@gmail.com>

* fix bug in AddInputOutputOpsPass: check existence of key in HashMap(inferface_lbi2scope_sym_id)

* fix bug in add_input_output_ops_pass.cpp: use iterator to avoid search twice

* fix bug when tensor access _tensor_buffer_shapes_and_dtypes

* remove unnecessary header

* format

* throw RuntimeError & add test

* Update oneflow/api/python/utils/tensor_utils.cpp
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

* support consistent tensor numpy

* refine

* add 2d sbp test

* refine

* refine

* add __init__.py in modules

* fix bug

* add cuda test case

* fix bug, cpu only

* fix bug in test about boxing not support
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* construct_consistentTensor_by_numpy

* fix typo

* fix error

* auto format by CI

* update get-oneflow
Co-authored-by: Nliufengwei <2472937968@qq.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Ntsai <jackalcooper@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* multiprocessing dataloader

* update

* update

* refine

* refine

* refine testcase

* refine

* refine

* refine

* export initNumpyCAPI

* refine

* refine

* refine

* refine

* refine

* refine

* SoftSyncNNGraphBuffers

* remove reduandent SoftSyncNNGraphBuffers

* simplify MakeXXXCriticalSection by SoftSyncNNGraphBuffers

* WaitUntilZero

* fix dead lock bug in LaunchLazyJob

* remove move assign constructor of WaitUntilZeroInstructionType
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix 2d-sbp consistent tensor initalization

* refine test

* refine 2d-sbp consistent tensor initalization

* refine & add check

* fix a bug
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check dtype in requires grad

* support more float dtype

* fix unittest dtype

* remove std cout

* fix

* fix autotest requires grad

* fix unittest

* enlarge test_rnn tolerance
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Implement device generator getstate and setstate.

* save/load auto generator state

* fix clang check

* fix clang check

* fix check

* fix unittest

* Fix generator in ddp, sync if state to be changed

* Remove unused code

* update random unittests

* revert util.cpp

* Fix
Co-authored-by: NBowen Chen <bob2420083992@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add sbp and placment doc

* refine test

* refine

* Update env_util.py

* Update python/oneflow/framework/docstr/placement.py
Co-authored-by: NYao Chi <later@usopp.net>

* Update sbp.py

* Update python/oneflow/test/modules/test_functional_docstr.py
Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: NYao Chi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* start up of framework

* refine

* refine

* add test on reset_docstr
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Replace the Send Receive of the buffer

* fix error
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* optional: refactor & make it a first-class user of JUST macros

* fix typo

* optional: init scalar var

* maybe: fix JUST macro

* maybe: fix name

* optional: add nullopt & remove explicit and error()

* optional: fix typo

* JUST: move to just.h

* optional: fix

* optional: format & fix

* optional: fix

* optional: add more test

* optional: fix

* forall: remove .value()

* forall:remove .value()
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* replace ##__LINE__
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* auto format by CI
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

* NNGraph input/output valid by register tensors

* LazyJobInstrutionType skip send Push/PullCB by NNGraph io valid

* Add test script

* Fix bug of static vec empty

* fix bug of clang
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* feat(Tensor): support assign copy interface

* feat(Module): use assign copy

* test(Tensor): add test for tensor assign_copy

* fix(Optim): create buffer data in step()

* rename assign_copy to set_data

* fix(Parameter): fix Parameter set_data bug

* refine log to log_once in data()

* test(ConsistentTensor): add set_data test

* fix(Module): fix Module.to_consistent bug

* auto format by CI
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* style(*): rename flow to oneflow in user hint

* fix(*): fix doctest

* auto format by CI

* remove ddp speed test
Signed-off-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

Signed-off-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add tensor yaml, support export tensor functional api.

* refine

* Remove packed functor signature

* remove unused file

* Refine

* refine

* add activation op import

* reinit oneflow init.py

* add oneflow abs and exp

* add oneflow abs and exp

* add acos

* add arccosh

* add more op

* add more ops

* add more op

* add more ops

* add log1p

* add more smaples

* add more ops

* add more ops

* add more ops

* add more ops

* Complete tensor functional apis.

* Fix pybind call

* add more ops

* add ops done

* Add target of_functional_tensor_obj

* Disable throw visibility warnings

* fix target link

* fix

* fix incorrect use of flow.Tensor.

* Fix error merge

* fix

* fix add unittest

* refine

* refine

* fix

* fix

* add tensor doc

* auto format by CI

* refine

* Fix

* Add doc for python function

* refine

* add tensor method docstring

* fix some bug

* fix docs bug

* Fix

* auto format by CI

* Tensor->tensor

* Tensor->tensor

* refine Tensor->tensor

* fix

* fix

* fix

* fix conflict

* fix bug

* fix ci bug

* fix

* delete diag op

* fix conflict

* Fix segment

* fix

* merge

* merge

* fix docstr bug
Co-authored-by: Nhjchen2 <chenhoujiangcug@gmail.com>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add tensor yaml, support export tensor functional api.

* refine

* Remove packed functor signature

* remove unused file

* Refine

* refine

* Complete tensor functional apis.

* Fix pybind call

* Add target of_functional_tensor_obj

* Disable throw visibility warnings

* fix target link

* fix

* fix incorrect use of flow.Tensor.

* Fix error merge

* fix

* fix add unittest

* refine

* refine

* fix

* fix

* add tensor doc

* auto format by CI

* refine

* Fix

* refine

* Fix

* auto format by CI

* fix

* fix

* fix

* fix

* Fix segment

* fix

* fix graph test
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add data_type bfloat16

* ifedf cu11

* add cuda_pseudo_bfloat16

* run unittes if cuda_version > 11000

* refine

* merge master

* merge master

* refine

* support flow.cuda.is_available

* export api docs

* refine

* fix docs

* format

* fix comments

* export cuda.device_count

* auto format by CI

* Update python/oneflow/cuda/__init__.py
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

* refine

* refine
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* GetBroadcastGroup

* fix comment typo.

* broadcast shape and dtype

* 1) rm THREAD_LOCAL_CACHED; 2) fix bugs in ThreadLocal

* fix wrong use of LocalRank

* 1) a decorator for disabling recursive boxing call; 2) a decorator for checking consistent tensor meta.

* don't set consistent_id when recursively calling eager consistent op interpreter.

* decompose nd_sbp boxing

* disable checking consistent tensor meta recursively.

* GetDecomposableEquivalent

* fix a unittest case bug

* fix a bug in unittest

* fix compiler complain

* add unitests for CalcDecomposableEquivalentShapeAndNdSbpPair

* InitNdSbpValidTransformationAxisSequence

* DecomposeIntoNaiveTransformations

* fix compiler complains

* move several unitests in parallel_desc_test.cpp into placement_sbp_util_test.cpp

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* refactor functional::ToConsistent

* refactor GetNdSbp

* fix compiler complaints

* upgrade gtest and fix static check error

* update head file index

* fix bug

* modify path of gtest lib

* refactor NaiveNdSbpBoxingInterpreter to BoxingExpr(symmetric-nd-sbp-to-nd-sbp)

* fix compiler complaints

* Update gmock_headers.txt

* Update gtest_headers.txt

* fix bug about disable checking consistent meta in local to consistent functor

* fix include bug
Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Nliufengwei <2472937968@qq.com>
Co-authored-by: NTwice <i@twice.moe>
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>

* support consistent_tensor.to(copy=True)

* auto format by CI

* refactor _consistent_tensor_to

* let cur_rank_phy_tensor be function::Empty(...) if !parallel_id.has_value()

* auto format by CI

* no OF_ENV_BARRIER when sync vm

* auto format by CI
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* disable_print_stack_in_tensor_numpy

* auto format by CI
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* move numpy c api init in numpy.cpp, make np array contiguous before copied into ofblob
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* fix bug, add test
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* fix type case, reformat
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* return error by maybe
Signed-off-by: Ndaquexian <daquexian566@gmail.com>

* fix error

* fix unimplemented to unimplementederror

* fix
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* feat(*): support tensor and optimizer serialization

* test(SGD): test sgd state_dict and load_state_dict

* test(Optim): test optim save and load state

* docs(Optim): add documents

* fix(Optim): recover enable_clip_grad for ParamGroup

* refine load_state_dict/state_dict inferface and add pickle file test
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* OFRecord reader eager consistent

* ofrecord reader header_infered_before_compute true
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Fix treat_args_as_inlist logic

* Align pytorch interface

* auto format by CI

* Fix empty and reformat

* fix

* auto format by CI

* fix merge
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>