Former-commit-id: 984cb0bfee36a7dd8648b5304228f6c3f66ceb01

Former-commit-id: 667a975135c63b1e03b6d332059b1078677f80f8

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: e841d975

Former-commit-id: 4e71cac6

* add check in deconv

* add check

* fix format

* fix check in python

* fix deconv2d params
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 20348c38

* add bceloss

* optimize some steps

* add bce loss diff evaluate

* fix format

* update test script

* fix annotation

* rebuild test code

* fix gen_arg

* fix arg

* fix name

* add bceloss

* fix format

* fix format

* Update test.yml
Co-authored-by: Ndoombeaker <later@usopp.net>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Former-commit-id: e2709ccd

* rename UserOpAttrVal to AttrValue

* Scope::GetAttrValue

* add ssp variable proxy pass

* AddSspVariableProxy

* ssp_config_def.cpp

* merge config_def from master

* REGISTER_SCOPE_CONFIG_DEF

* description for ssp_partition_strategy

* fix return type of JobPass::HasState

* FlexDef/FlexValue

* support recursive flex def

* remove field_number

* more cfg files

* instructions builder

* forward declaration instead of include

* more test for cfg

* revert cfg files

* InstructionsBuilder

* using std::function as argument of IdCache::FindOrCreate

* scope op_collection

* include <functional> in framework/interpreter.h

* puts more code into WithOptimizerOpCollectionScope

* IsInOptimizerOpCollection

* include <functional> in symbol_id_cache.h

* calculation pass

* IsInOptimizerOpCollection -> IsInOptimizerPass

* minor refine about spp_config_def.cpp

* test for add_ssp_variable_proxy

* rm framework/flex

* refine add ssp variable proxy pass

* refine Error

* refine Error

* AddScopeToPyStorage

* fix test_watch

* get scope_symbol_id from current scope

* fix assert bug

* no longer use scope_proto.symbol_id
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: ef6786d5

* more cfg files

* instructions builder

* forward declaration instead of include

* more test for cfg

* revert cfg files

* InstructionsBuilder

* using std::function as argument of IdCache::FindOrCreate

* scope op_collection

* include <functional> in framework/interpreter.h

* puts more code into WithOptimizerOpCollectionScope

* include <functional> in symbol_id_cache.h

* calculation pass

* refine Error

* AddScopeToPyStorage

* fix test_watch

* get scope_symbol_id from current scope

* fix assert bug
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 1467aecf

* multi_count_not_finite op

* refine

* add cpu

* refine

* refine

* refine

* format

* add count not finite

* fix
Co-authored-by: Nguoran <guoran@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Nliujuncheng <liujuncheng1022@gmail.com>
Former-commit-id: 58627eec

Co-authored-by: Nguoran <guoran@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: ab5cde94

* Simple kernel memory bandwidth profiler

* Add new line

* refine

* StringToBool support yes/on/y

* use callback

* fix WITH_CUDA
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: ec9de690

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 0e5ad637

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 24c2c931

* 优化 rmsprop optimizer.

* lars optimizer

* bug fix

* reimplement the function of rmsprop.

* delete folder CMakeFiles.

* adjust the code according to advices from reviewers.

* add mean_gradient according to parameter centered.

* remove the annotation of unit test script.

* exchange the position of 'T* model' and 'T* momentum'.

* rewrite the RmsPropUpdateFunctor.

* modify the code according to review advices.

* arrange the format of the code.
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 7c34e35a

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: a0051e66

* draft

* test sort op

* argsort

* argmax

* quick imple for top_k, to be tested

* feat

* test argmax

* pass all tests

* fix type

* batch_axis_non_change true

* final

* format

* refine by review

* accurate axis check

* Update oneflow/python/ops/transpose_util.py
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>
Co-authored-by: NMardino <42901638+MARD1NO@users.noreply.github.com>
Co-authored-by: NYinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: a29c228c

* Replace the py instruction with CFG Instruction

* move RunInstruction to pybind & refactor EagerOneflow's interface by cfg

* use forward declaration

* fix code style

* move RunInstruction api to oneflow_api.vm

* remove useless line in oneflow_internal.i

* replace args str to cfg_obj in python callback

* add forward declear of InstructionListProto

* cancel forward declear of InstructionListProto

* fix a name spelling mistake

* add TODO and use const reference instead of variable

* fix code style

* use const reference
Co-authored-by: Nouyangyu <xuanjiuye@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 53282de0

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 282d3579

* return func updated for operator==

* cfg operator== is reformed

* exceptions fixed

* switch expression converted

* cfg operator< reformed

* formated

* uesless brackets removed

* format updated
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 99e40878

* use_union_instead_of_struct_for_oneof

* 64-bit aligned and remove shared_ptr.reset() & clear
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: ac9efc52

* py op draft

* naive py op

* add grad

* add fake test

* add pybind make

* add cmake

* refine cmake

* refine pybind cmake

* add python lib

* user pybind11 inner cmake

* simple pybind11 cmake

* fix conflict of _GLIBCXX_USE_CXX11_ABI, add py-dev lib, add pybind11 auto download compile pass

* use py-c-api

* numpy c api compile pass

* draft passed demo test

* uni_interface_draft

* add input

* add part of output

* add numpy type to of type

* add output

* add log

* general numpy input & output test pass

* rename numpy converter

* add py op register draft

* add wishful code

* finish py op compute

* add grpc error log

* add test of sigmoid_grad

* get input output tensor by definition order

* fix python interpreter memory leak

* fix format

* add multioutput test

* restore conv2d

* fix format

* refine test

* refactor py op compile

* refine python op compile draft

* add python compile & test

* add test

* fix load so

* single test pass

* refien op lib compile api

* refactor lib name

* api add by lib

* supprot lib folder

* support lib with structure test pass

* refactor one2two

* rm useless code

* fix dir bug

* revert cmake

* format

* rm useless code

* export api

* format

* test file path

* support lib path

* export BlobDef&unique_str

* add cp of cpp2py for whl

* rm useless code

* debug ci g++ abi

* fix format

* debug ci cxx abi

* debug ci docker : close c++11 abi

* revert cxx abi set

* CI compile so debug

* CI compile so debug2

* CI compile so debug3

* rm debug code&deal with eagar ci

* refine test

* detect_cxx11_abi_availibility

* crash on compiling error

* rm BlobDef

* var to func

* support switch python and cpp kernel

* py cpp kernel templete rm type

* fix compile

* debug ci

* fix imp.find_module warning, restore test

* attr use new interface

* change test

* add test log

* try to fix test in ci

* add ci test debug log

* ci add special test for op build

* fix format

* refine style

* py kernel use cache

* op_lib op_lib_loader to experimental

* load so now, new py kernel reg cpp api

* refine

* py_ext compile pass

* dynamic compile to static compile, test pass

* rename test case

* user python kernel

* op_lib_reg -> op_lib_builder & refine api

* refine code and test pytorch in kernel

* no link python so

* fix last line

* test op load in eager and fix style

* py kernel import only once and use sub-namespace pyext

* fix last line

* fix lib path

* refine op lib api

* rm stringstream
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 14fe039a

* add cpu version oneflow_range

* fix name

* add test case and fix assert

* fix bug and add test case

* add cuda kernel of range

* add gpu test version

* add gpu test option

* fix test case

* fix format

* reconstruction oneflow_range

* add double type and add new test case

* remove range shape attr

* use xpu1dkernel_loop

* add one device two nodes test

* add make format

* add float16 failure version

* refine

* still have error

* refine  still have error in float16

* fix link problems (a lot work left to refine)

* refine as operator version, add new test cases

* fix to int32_t, SBP still in process

* fix format

* fix name and int32_t

* fix test case

* fix of_format
Co-authored-by: Ndoombeaker <later@usopp.net>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: a3c48e81

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 460d257a

* merge all ci jobs into one

* refine

* refine

* rename

* add extra_flags

* add quotation

Former-commit-id: 74ecce5d

* add l1 loss op in python

* add l1 loss test

* add reduction and new test cases

* add mean diff test case

* delete useless annotation and add a check of reduction type

* add equation in doc

* fix name

* fix 1n2d test

* fix format

* delete useless grad

* fix arg

* refine

* fix name
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 8a35cd87

Former-commit-id: 1fa63b56

Former-commit-id: fccc1cda

Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: eceb09a0

* startup of developing of torch.gather

reference:
https://pytorch.org/docs/stable/generated/torch.gather.html

* add end blank line

* segment fault to be fixed

* add end line

* torch gather cpu version(forward)

* refine cpu kernel

* refine cpu version(comment and test case)

* fix

* GPU kernel

* remove sparse grad attr

* finish gather dim forward GPU version

* start up of scatter_dim op

* finish scatter dim add op(both cpu and gpu version)
 which needed by gather dim's backword process

* support float16(half) for scatter_add_dim on gpu

* rename files, use gather_dim instead of torch_gather

* refine cpu/gpu kernel, remove redundancy

* test_troch_gather.py -> test_gather_dim.py

* add CHECK expr in op

* reformat

* add InputArgMoifyFn (index grad not required)

* register sbp

* rename some prarams

* refine (use functor to reduce code redundancy)

* fix gahter_dim index bug; fix bug in nd_index_slice_ops.cpp

* refine test case and fix bugs

* refine test case

* refine (grad problems on gpu left)

* refine test_case(grad problem unsovled)

* fix gpu grad kernel bugs

* add arg struct for ndindexHelper

* replace CoordinateConverter by NdIndexOffsetHelper

* add split axis check (split_axis != dim)

* fix 1n2cards bugs (wrong sbp siganture)

* refine test case

* format code

* rename file name(use dim_gather instead of gather_dim)

* rename(use dim_gather) and add docstring

* fix tyops

* half support

* rollback array_ops.py

* float16 grad crashed

* fix half bugs(cuda version)

* fix DeviceAdd general template

* refine var name, macro reuse

* Input 'src' -> 'input'

* rename op Inputs

* refine test case

* refine files hierarchy

* remove unneed header files

* refine DimGatherFunctor, remove useless argument member

* refine DimScatterAddFunctor, remove useless argument member

* refine

* refine register kernels

* reformat

* float16 only works on CUDA 10.2

* reformat

* refine - ctx arg first

* refine test case

* remove float16 test

* refine

* refine: use int32_t dim, simplfy tensor shape conversion

* locate half bug, remove float16 test case

* reformat

* refine

* add ifdef WITHCUDA for cu

* reformat

* refine (remove useless macro)

* refine (cpu only test)

* fix cpu-only problems

* add cpu-only decorator for test case

* refine typos and add const quialifier

* refine op check

* refine test case

* refine op check

* remove global delcration in test case

* refine depending on comments of WangYinggang

* refine batchaxis

* refine batchaxis

* use np.take_along_axis instead of _np_gather

* refine _np_dim_scatter_add(use ravel_multi_index unravel_index)

* fix typos
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: a94c2f5b

* update affiliations to team up 11 and 13

* use ubuntu-18.04 rather than latest

Former-commit-id: 8e7c99a5

Former-commit-id: 91396f09

* more cfg files

* instructions builder

* forward declaration instead of include

* more test for cfg

* revert cfg files

* InstructionsBuilder

* using std::function as argument of IdCache::FindOrCreate

* include <functional> in framework/interpreter.h

* include <functional> in symbol_id_cache.h
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: f915261f

* Update README.md

* Update README.md

* Update test.yml

* Update test.yml

* Update action.yml

Former-commit-id: b9cfed88

Former-commit-id: db239cc9

Former-commit-id: 14392a0c

* dont check label on sync

* rm pull_request_target

Former-commit-id: 12c8c8ef

Former-commit-id: 26097f29

* Replace the py instruction with CFG Instruction

* move RunInstruction to pybind & refactor EagerOneflow's interface by cfg

* use forward declaration

* fix code style

* move RunInstruction api to oneflow_api.vm

* remove useless line in oneflow_internal.i

* add forward declear of InstructionListProto

* cancel forward declear of InstructionListProto
Co-authored-by: Nouyangyu <xuanjiuye@gmail.com>
Former-commit-id: 780f2b0b

* run ci when add label need_ci

* change name

* request review from bot

* add reveiw from bot for fmt

* use bot name oneflow-ci-bot

Former-commit-id: 2d2b12c5

Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Former-commit-id: 3ce5e436