Former-commit-id: 809793c4

Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 0a85024c

* add docs to env_util.py function_util.py

* add docs to config_util.py

* add docs to session_util.py

* add docs to layer.py

* add docs to layers.upsample_2d

* add docs nn_ops

* format refine

* modify name_scope.py

* add docs >> placement_util.py

* add docs to distribute.py

* add docs to data_ops.py

* add docs to user_op_builder.py

* add docs to math_ops.py

* fix conflict

* add docs to initializer_util.py

* refine docstring

* Final Cut (#3354)

* bump version

* rm pip install
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>

* remove windows directory (#3358)

* install python pkgs from dev-requirements.txt in configure time, remove PY3 flag (#3351)

* install python pkgs from dev-requirements.txt in configure time

* remove PY3 cmake flag, add dev-requirements.txt to CMAKE_CONFIGURE_DEPENDS, remove unused cmake functions

* build nccl with -jN

* special tag for gpu (#3362)
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

* ci master (#3366)
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Co-authored-by: NOuYang Yu <xuanjiuye@gmail.com>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Former-commit-id: 8b07d01e

* wip

* wip

* remove instruction2, make Run() method empty

* use HostStreamType instead of CpuStreamType

* get blob by lbn and parallel_id, separate cpu and gpu instructions

* fix wrong initialization order

* set parallel_ctx field of regst_desc in task::NewProducedRegst

* refine code

* rename MaybeRun -> Run

* reformat

* keep the track of regst_desc_id and parallel_ctx in register manager

* add license

* use lbi instead of lbn

* variable op -> interface op

* set is_mutable to true in other interface ops

* commit debug code

* add api in flow.experimental, record job_name for interface ops, pass job_name to BlobDef and EagerConsistentBlob

* add lock on map concurrent write

* clean

* keep the track of lbi&parallel_id and blob* in register manager

* remove hardcoded 'out'

* update comments

* use right dtype, add test

* add missing sync

* add type annotations in test job

* support mirrored and tensor list blob

* wrap cuda_blob_instruction_type.cpp with #ifdef WITH_CUDA, skip the test in cpu only build
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: b5b48550

* add back bert cpu

* Revert "add back bert cpu"

This reverts commit 417684af4944fb6c4a24c6417a907407f3dc4972 [formerly a4bac19eabe2d5242c2d8ece6ca42569a9c34caa].

* only  test_inplace  because cpu bert is too slow
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 71b1ef44

Former-commit-id: 5fc44b2e

Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: befc5353

Former-commit-id: e7c72d39

* Fix cudnn conv math type fallback

* refine

Former-commit-id: 21145f99

Former-commit-id: 968dc8e6

* refactor image resize part 1

* rm resize_shorter

* interpolation

* change target resize

* swap h and w position in size and scale

* update python api and test

* refactor op & kernel

* unify api

* fix by self review

* fix test case

* add new image_resize test script

* fix merge

* Final Cut (#3354)

* bump version

* rm pip install
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>

* remove windows directory (#3358)

* install python pkgs from dev-requirements.txt in configure time, remove PY3 flag (#3351)

* install python pkgs from dev-requirements.txt in configure time

* remove PY3 cmake flag, add dev-requirements.txt to CMAKE_CONFIGURE_DEPENDS, remove unused cmake functions

* build nccl with -jN

* special tag for gpu (#3362)
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

* ci master (#3366)

* fix merge

* image_resize backward compatible

* revert new_data_pre

* backward compatible old api return

* minor fix

* update test

* fix api_image_resize return annotation

* fix mistake spell

* image_resize support converting to dtype

* rm ImageResizeToFixedSizeKernel template T

* doc for api_image_resize

* correct image_resize doc

* image_resize_keep_aspect_ratio all output tensor buffer

* tensor buffer to tensor in api_image_resize
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Co-authored-by: NOuYang Yu <xuanjiuye@gmail.com>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 22bb94a6

* larger rol for conv 2d cpu

* larger tol and print diff

Former-commit-id: e54ff95e

Co-authored-by: Nzjlab <2496533749@qq.com>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 06da1a03

* skip known issues

* add return

Former-commit-id: a82c3e03

* refactor slice user op to support cpu

* support half

* update api

* resolve full specialization problem with explicit instantiation

* fix bugs

* add typing.List to return annotation

* fix slice grad bug

* remake slice test

* fix bug and add new test case

* add reverse test

* fix compile warning

* support register with_cuda

* support ci build_and_test_cpu

* add fp16 test case

* improve perf on gpu using static NDIM

* comment debug print

* fix old api

* change by review comment

* fix bugs

* use SliceOffsetToEntireOffset<NDIM> in cpu SliceKernelUtil

* change by review

* fix

* fix

* update by merge
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: f10d10e9

* refactor message OperatorConf, change device_type to device_tag

* subsititute HobDeviceType with HobDeviceTag in user_op kernel registration

* remove c_api_util.DeviceType4DeviceTag

* fix error when buil with cuda off

* fix can not use CHECK_JUST macro in another macro

Former-commit-id: 120a7f86

* rm trainable in if

* add ln cpu and refactor bn

* refactor bn

* do affine in LN nor BN

* fix typo

* fix typo

* fix typo

* add NotImplementedError

* add nn test cpu

* refactor nn bn

* fix test case

* update log

* rename axes

* check in moments

* rm print

* fix range

* add log

* fix typo

* add log

* fix bugs

* Dev bn training cpu (#3445)

* enable trainable_without_training for cpu

* add training
Co-authored-by: Ntsai <caishenghang@oneflow.org>

* run cpu gan

* Revert "run cpu gan"

This reverts commit 5633d0d2042fd0c4017c02570c7d0d258c8b1856 [formerly e98deecf06f5ad3e50ad42993ba5b7149d4e1642].

* fix order

* optional normalized

* larger tol when cpu
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 4d44113e

Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Former-commit-id: d0abcfb5

* add oneflow.typing.Collection

* add test script of oneflow.typing.Collection

* fix code style

* simplify code

* remove notes

* fix code style

* remove code useless

* simplified code

* Decoupling container transform precess

* remove brackets

* rename container to bundle

* fix code style

* add docstring

* fix docstring

* add docstring
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: bce2b3c9

Former-commit-id: 5b2a2f2f

Former-commit-id: a17ab4de

Former-commit-id: 8acd1132

Former-commit-id: 63e48110

Former-commit-id: e3618206

* Add OF_CUDA_CHECK/OF_CUDNN_CHECK/OF_CUBLAS_CHECK/OF_CURAND_CHECK

* OF_NCCL_CHECK

* use OF_CUDA_CHECK

* use OF_CUDNN_CHECK

* use OF_CURAND_CHECK

* use OF_NCCL_CHECK

* use OF_CUBLAS_CHECK

* use OF_CUDNN_CHECK

* use OF_CUBLAS_CHECK

* OF_CUDA_CHECK

* OF_CUDA_CHECK

Former-commit-id: 142b8622

* image.random_crop

* refine code for review

* refine code for review

Former-commit-id: 3bdc1a44

Former-commit-id: aac072df

* ofrecord_image_classification_reader

* refine

* use num_decode_threads_per_machine

* Specify image_feature_name and label_feature_name

* Extract LoadWorker and DecodeWorker

* Remove image_feature_name/label_feature_name default value

* underlaying_=>base

* set decode_buffer_size_per_thread to 32

Former-commit-id: 526766a7

Former-commit-id: 50b467cf

* rm blank

* add more log

* support bn cpu inference
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 96263615

Former-commit-id: d2c765ad

* cmake dont panic when build cuda

* naive changes

* fix cudaMemcpyKind

* fix acc actor

* fix actor

* fix gdb

* fix vm

* fix work type

* fix cuda type

* fix cuda type

* fix collective backend

* fix global scope

* amp

* rm PackKernelUtil gpu

* fix log

* fix rand

* fix sync size

* fix allocator

* fix vm

* fix kernel

* fix kernel

* fix kernels

* fix kernel

* fix softmax

* fix kernels

* fix reshape kernels

* add workaround

* try fix symbol not found

* fix vm

* fix vm

* fix jpeg

* fix broadcast gpu

* fix broadcast like

* fix transpose

* fix matmul

* fix CopyElemOnGpu

* fix sigmoid

* fix sigmoid and softmax

* fix relu

* fix sparse cross entropy

* fix kernels

* fix tanh

* fix same padding

* fix softmax

* fix undefined symbol: gzgets

* fix CopyField

* fix scalar add

* fix CopyNDGpuImpl

* copier

* fix slice boxing

* fix mem copier

* fix zero like

* fix acc actor

* fix dev pure cpu (#3410)

* add pure cpu message

* add default for src dir in ci

* add cpu workflow

* fix extra_oneflow_cmake_args

* fix link problem add update readme

* fix _GetDefaultConfigProto

* use gpu

* rm default value for gpu dev num

* rm cpu ci

* check in skip code

* refine ci and add back

* add arg for appendix

* fix env arg

* fix env arg

* rm make nccl

* add back to pass gpu ci

* get with cuda in py

* fix fmt

* fix test kernels

* fix activation

* add skips

* fmt

* skip all reduce

* fix assign

* fix bn

* fix bn

* rm tmp_wheel

* rm tmp_wheel

* move clean up

* for CPU-only OneFlow make gpu_device_num equivalent to cpu_device_num

* change warning to info

* only run 3 iters for cpu

* skip bert for cpu

* fix check

* add warning

* add cpu Integration test

* Dev pure cpu test cases (#3422)

* fix test cpu cases

* change os.getenv('ONEFLOW_TEST_CPU_ONLY') == 'True' to os.getenv('ONEFLOW_TEST_CPU_ONLY')

* print traceback for info and warning

* fix test cpu cases

* add more info on why skip check of resnet

* skip gan in cpu
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Co-authored-by: NJackieWu <wkcn@live.cn>
Co-authored-by: NOuYang Yu <xuanjiuye@gmail.com>
Former-commit-id: 47aba214

Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: e5e3eb31

Former-commit-id: 438c45a5

1. add link
2. add item in table of contents
Co-authored-by: NYao Chi <yaochi@oneflow.org>
Former-commit-id: 1302e01a

Former-commit-id: 64c9298d

* add conv1d conv3d

* add transpose in conv3d NDHWC

* fix

* fix

* NCHW->NCDHW
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 6304f8a2

Former-commit-id: e65f5473

* Add tensor_buffer_to_tensor and tensor_to_tensor_buffer

* const auto& => const auto

Former-commit-id: 3bd90160

Former-commit-id: 59d22372