Former-commit-id: 1f42431a

* fix load mnist data

* format
Co-authored-by: Nguo ran <360112263@qq.com>
Former-commit-id: 818d50d2

* Fix do_bias_correction for Adam

* refine

Former-commit-id: ad1aac6e

* bump version 0.1.9

* update readme

* update script

* add cuda 11 oneflow

* add cpu install

* add cuda 11 script

* add cpu only instruction
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Former-commit-id: 297e636f

* add api docs

* fix docs

* add api doc in math_ops

* Change the theme to read_the_docs

* add equation docs in math_add

* add equation docs in sub, mul, addn

* add mod div tanh docs

* add gelu, relu docs

* add sigmoid unsorted_segment_sum api docs

* add unsorted_segment example docs

* add equal to greate_equal example docs

* add logical and example docs

* add other example docs, include squared difference

* add requirements of sphinx_rtd_theme

* delete the requirement of sphinx-material

* use make of format
Co-authored-by: Ndoombeaker <later@usopp.net>
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 3a1ef10f

Former-commit-id: 2ec8fc69

* add SubTskGphBuilderUtil::BuildBoxingInfo() interface

* add BuildBoxingInfo to sub_task_graph_builder

* save boxing log

* add not nullptr check

* add TYR

* remove blank line

* change BuildBoxingInfo to BuildBoxingLogInfo

* fix code style

* rename boxing_logging_ to boxing_logging_lines_

* fix c++ code format

* rename boxing_info to boxing_log_line

* rename boxing_info to boxing_log_line

* fix code format

* abstract SubTskGphBuilderStatus struct

* rename boxing_log_line to boxingbuilderstatus

* delete useless code

* fix code format

* remove useless include file

* complete GetBlobInfo4LogicalBlobDesc

* change boxing_log_lines to boxing_log_list_(std::list)

* Optimize log saving function

* remove useless include file

* add colnum filed  Macro

* add CHECK_NOT_NULL

* Overload SetLogStream

* change inappropriate variable name

* remove usleless include file

* add CHECK_JUST & TRY

* fix c++ format

* add new line at end of file

* optimization boxing log

* remove useless include

* fix c++ format

* optimize SubTskGphBuilderStatus Getters

* change macro name

* fix c++ format

* optimize boxing log

* fix c++ format

* rename SerializeParallelDesc to ParallelDescToString

* optimize boxing log

* fix small bug

* optimize boxing log

* fix c++ format

* optimize boxing log

* fix c++ format

* optimize boxing log

* fix c++ format

* optimize boxing log

* optimize boxing log

* fix c++ format

* optimize boxing log

* remove useless include file

* optimize boxing log
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: e0370ccd

* move generated python files into oneflow/python_gen

* python_gen.framework.sysconfig -> python_gen.sysconfig

* write file in build step instead of configure step

Former-commit-id: 1d35fa1e

* conv2d cpu groups

* address review
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 79256ae2

Former-commit-id: 0fb58c4d

* check in debug code

* add color code

* attr working

* different color

* rm debug code

* quick fix

* color task graph

* color edges

* rename
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: d4475d86

* create symlink instead of copying python code

* delete the old python dir
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 51113e68

* simplify script and fmt tools

* fix ci

* add ll

* fix root edit file

* use docker root

* w for all users

* fix cpu

* add back clean

* rm debug print
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Former-commit-id: af8102a1

Former-commit-id: 73b3905c

Former-commit-id: 809793c4

Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 0a85024c

* add docs to env_util.py function_util.py

* add docs to config_util.py

* add docs to session_util.py

* add docs to layer.py

* add docs to layers.upsample_2d

* add docs nn_ops

* format refine

* modify name_scope.py

* add docs >> placement_util.py

* add docs to distribute.py

* add docs to data_ops.py

* add docs to user_op_builder.py

* add docs to math_ops.py

* fix conflict

* add docs to initializer_util.py

* refine docstring

* Final Cut (#3354)

* bump version

* rm pip install
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>

* remove windows directory (#3358)

* install python pkgs from dev-requirements.txt in configure time, remove PY3 flag (#3351)

* install python pkgs from dev-requirements.txt in configure time

* remove PY3 cmake flag, add dev-requirements.txt to CMAKE_CONFIGURE_DEPENDS, remove unused cmake functions

* build nccl with -jN

* special tag for gpu (#3362)
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

* ci master (#3366)
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Co-authored-by: NOuYang Yu <xuanjiuye@gmail.com>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Former-commit-id: 8b07d01e

* wip

* wip

* remove instruction2, make Run() method empty

* use HostStreamType instead of CpuStreamType

* get blob by lbn and parallel_id, separate cpu and gpu instructions

* fix wrong initialization order

* set parallel_ctx field of regst_desc in task::NewProducedRegst

* refine code

* rename MaybeRun -> Run

* reformat

* keep the track of regst_desc_id and parallel_ctx in register manager

* add license

* use lbi instead of lbn

* variable op -> interface op

* set is_mutable to true in other interface ops

* commit debug code

* add api in flow.experimental, record job_name for interface ops, pass job_name to BlobDef and EagerConsistentBlob

* add lock on map concurrent write

* clean

* keep the track of lbi&parallel_id and blob* in register manager

* remove hardcoded 'out'

* update comments

* use right dtype, add test

* add missing sync

* add type annotations in test job

* support mirrored and tensor list blob

* wrap cuda_blob_instruction_type.cpp with #ifdef WITH_CUDA, skip the test in cpu only build
Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: b5b48550

* add back bert cpu

* Revert "add back bert cpu"

This reverts commit 417684af4944fb6c4a24c6417a907407f3dc4972 [formerly a4bac19eabe2d5242c2d8ece6ca42569a9c34caa].

* only  test_inplace  because cpu bert is too slow
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 71b1ef44

Former-commit-id: 5fc44b2e

Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: befc5353

Former-commit-id: e7c72d39

* Fix cudnn conv math type fallback

* refine

Former-commit-id: 21145f99

Former-commit-id: 968dc8e6

* refactor image resize part 1

* rm resize_shorter

* interpolation

* change target resize

* swap h and w position in size and scale

* update python api and test

* refactor op & kernel

* unify api

* fix by self review

* fix test case

* add new image_resize test script

* fix merge

* Final Cut (#3354)

* bump version

* rm pip install
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>

* remove windows directory (#3358)

* install python pkgs from dev-requirements.txt in configure time, remove PY3 flag (#3351)

* install python pkgs from dev-requirements.txt in configure time

* remove PY3 cmake flag, add dev-requirements.txt to CMAKE_CONFIGURE_DEPENDS, remove unused cmake functions

* build nccl with -jN

* special tag for gpu (#3362)
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>

* ci master (#3366)

* fix merge

* image_resize backward compatible

* revert new_data_pre

* backward compatible old api return

* minor fix

* update test

* fix api_image_resize return annotation

* fix mistake spell

* image_resize support converting to dtype

* rm ImageResizeToFixedSizeKernel template T

* doc for api_image_resize

* correct image_resize doc

* image_resize_keep_aspect_ratio all output tensor buffer

* tensor buffer to tensor in api_image_resize
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Ntsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Co-authored-by: NOuYang Yu <xuanjiuye@gmail.com>
Co-authored-by: Ndaquexian <daquexian566@gmail.com>
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 22bb94a6

* larger rol for conv 2d cpu

* larger tol and print diff

Former-commit-id: e54ff95e

Co-authored-by: Nzjlab <2496533749@qq.com>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 06da1a03

* skip known issues

* add return

Former-commit-id: a82c3e03

* refactor slice user op to support cpu

* support half

* update api

* resolve full specialization problem with explicit instantiation

* fix bugs

* add typing.List to return annotation

* fix slice grad bug

* remake slice test

* fix bug and add new test case

* add reverse test

* fix compile warning

* support register with_cuda

* support ci build_and_test_cpu

* add fp16 test case

* improve perf on gpu using static NDIM

* comment debug print

* fix old api

* change by review comment

* fix bugs

* use SliceOffsetToEntireOffset<NDIM> in cpu SliceKernelUtil

* change by review

* fix

* fix

* update by merge
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: f10d10e9

* refactor message OperatorConf, change device_type to device_tag

* subsititute HobDeviceType with HobDeviceTag in user_op kernel registration

* remove c_api_util.DeviceType4DeviceTag

* fix error when buil with cuda off

* fix can not use CHECK_JUST macro in another macro

Former-commit-id: 120a7f86

* rm trainable in if

* add ln cpu and refactor bn

* refactor bn

* do affine in LN nor BN

* fix typo

* fix typo

* fix typo

* add NotImplementedError

* add nn test cpu

* refactor nn bn

* fix test case

* update log

* rename axes

* check in moments

* rm print

* fix range

* add log

* fix typo

* add log

* fix bugs

* Dev bn training cpu (#3445)

* enable trainable_without_training for cpu

* add training
Co-authored-by: Ntsai <caishenghang@oneflow.org>

* run cpu gan

* Revert "run cpu gan"

This reverts commit 5633d0d2042fd0c4017c02570c7d0d258c8b1856 [formerly e98deecf06f5ad3e50ad42993ba5b7149d4e1642].

* fix order

* optional normalized

* larger tol when cpu
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 4d44113e

Co-authored-by: NLi Xinqi <lixinqi2010@gmail.com>
Former-commit-id: d0abcfb5

* add oneflow.typing.Collection

* add test script of oneflow.typing.Collection

* fix code style

* simplify code

* remove notes

* fix code style

* remove code useless

* simplified code

* Decoupling container transform precess

* remove brackets

* rename container to bundle

* fix code style

* add docstring

* fix docstring

* add docstring
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: bce2b3c9

Former-commit-id: 5b2a2f2f

Former-commit-id: a17ab4de

Former-commit-id: 8acd1132

Former-commit-id: 63e48110

Former-commit-id: e3618206

* Add OF_CUDA_CHECK/OF_CUDNN_CHECK/OF_CUBLAS_CHECK/OF_CURAND_CHECK

* OF_NCCL_CHECK

* use OF_CUDA_CHECK

* use OF_CUDNN_CHECK

* use OF_CURAND_CHECK

* use OF_NCCL_CHECK

* use OF_CUBLAS_CHECK

* use OF_CUDNN_CHECK

* use OF_CUBLAS_CHECK

* OF_CUDA_CHECK

* OF_CUDA_CHECK

Former-commit-id: 142b8622

* image.random_crop

* refine code for review

* refine code for review

Former-commit-id: 3bdc1a44