Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into dev_replace_ErrorProto_to_cfgErrorProto

Conflicts:
	cmake/cfg.cmake
	tools/cfg/template/template.cfg.h
	tools/cfg/util/proto_reflect_util.py

Former-commit-id: 64e6bae189d74bc92dbabed67753562415490d79

Former-commit-id: b77284af

* support reflecting cfg message

* replace python f-string with traditional format

* more comments for cfg::Message methods

Former-commit-id: 26059359

* integrate cfg primary

* add depend of of_ccobj

* add dependence of of_cfgobj

* fine-tuning the way of cfg integration

* add proto/cfg convert & DebugString

* finetune cfg

* optimize cfg & add support of nested message

* fix a small bug

* arrange cfg code

* optimize cfg

* fix code format

* add of_proto_python_dir arg to template_convert.py

* fix cfg constructor bug

* remove useless line in cfg.cmake

* optimize cfg.cmake

* delete useless line in cfg.cmake

* fintune cfg.cmake

* fix a small bug of oneof field

* support oneflow_api_registry

* fix code format

* Expand folder range

* optimize of_api_registry

* Update of_api_registry.h

add a blank line between constructor  and member function

* update pybind_module_registry

* fix code format

* optimize oneflow_pybind_api

* optimize pybind registry

* fix code foemat

* optimize cfg

* fix oneflow.cmake

* add a blank line

* optimize cfg
Co-authored-by: Nlixinqi <lixinqi0703106@163.com>
Former-commit-id: e01b57a5

Former-commit-id: aa4373bbb9119bb88dd182b748f68e03034b18e4

Former-commit-id: 86871384e3c06df8d280bfb46579a5123ae7001d

Merge branch 'dev_integrate_cfg' of https://github.com/Oneflow-Inc/oneflow into dev_replace_ErrorProto_to_cfgErrorProto

Conflicts:
	oneflow/python/oneflow_internal_helper.h

Former-commit-id: 2e3c9c969b2c58cc7573ad9f2273e44891461380

Former-commit-id: 310030e4b84f23456caf7c515696e59d05a8ccd2

* refactor Session.job_name2current_scope_ to Session.scope_stack_

* Seperate scope related functions from Session to scope_util.py

* revert change
Co-authored-by: NShenghang Tsai <jackalcooper@gmail.com>
Former-commit-id: 278b50ae

* a) refactor Operator::GetCustomizedConf; b) refactor ReplaceInputLbnInOpCustomizedConf();

* rm XrtLaunchOp::GetCustomizedConf

* refactor xrt/passes/rebuild_job_pass

Former-commit-id: 9d68b158

* a) remove oneflow.placement.current_scope; b) refactor PlacementScope

* refactor GlobalModePlacementScope

Former-commit-id: d23a3fca

Former-commit-id: 64c20462

Former-commit-id: 93c120f8

* Add CudnnConvAlgoCache

* refine

Former-commit-id: a2af59ea

Former-commit-id: ead15e3b

* Fused ImageDecoderRandomCropResize

* format

* fix cpu build

* Add comment

* dest=>dst

* Add comment

* Add ImageDecoderRandomCropResizeOp::InferParallelSignature (#3646)

* Add ImageDecoderRandomCropResizeOp::InferParallelSignature

* Add Scope::scope_proto

* refine

Former-commit-id: ecb64555

* bn_add_relu use bit mask

* refine

* refine

Former-commit-id: 315d6962

* MaskAndScaleHalf2

* fix

* fix

* refine

* format

Former-commit-id: 31701b01

* rename OF_BARRIAER

* add eager_2node_test.py

* InitLazyGlobalSession if eager execution not enabled

* remove Global<LbiDiffWatcherInfo>

* add TODO() comments for OF_SESSION_BARRIER under directory core/comm_network/

* remove eager_2node_test.py

Former-commit-id: 4d3e2d33

* add classes for different test base class

* add export

* rm old main files

* fix

* fix class name

* check in changes of tests

* fix else

* fix bugs

* fix order

* better naming

* update ci script

* fix import unittest

* add --failfast

* filter by card num

* fix typo

* update ci script

* dont set CUDA_VISIBLE_DEVICES

* use skip unless

* add OneGpuTestCase

* fix typo

* shorter reason

* fix class

* fix typo

* skip test_train_consistent for now

* add tearDown

* fix setup

* rm todo

* skip TestImageResize

* fix for pure cpu

* skip TestImageTargetResize

* fix typo

* rename

* mv demo

* rm readme

* rename

* rename cases

* add --verbose

* rename

* address reviews

* fix export

* minor fix

* fix export

* fix skips

* more renames

* rename

* rename class

* rm gpu

* rename num in scripts

* skip resize img

* fix sys env

* fix dynamic reshape

* fix dynamic reshape

* fix skip msg

* rm num_gpus

* address review

* address reviews

* rm class

* update decorator

* fix inden

* fix typo

* replace class with decorator

* fix

* refine

* fix decorator

* mv code

* add more decorators

* add ONEFLOW_TEST_DEVICE_NUM=1

* fix typo

* fix typo

* fix name

* fix skip msg

* use decorators in scripts

* larger rol for xla ln
Co-authored-by: Ntsai <caishenghang@oneflow.org>
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 789e80b6

* model update fuse scalar_mul_by_tensor

* add test case

* testcase

* fix

* refine

* fix

* refine

Former-commit-id: 5a2ed96c

* BiasAdd Row Col Half2

* refine

Former-commit-id: a141d6a2

* fused layer norm

* fix

* rm debug info

* refine

* fix

* testcase

* refine

* testcase

* refine
Co-authored-by: Nguoran <guoran@oneflow.org>
Co-authored-by: NJuncheng <liujuncheng1022@gmail.com>
Former-commit-id: 6e5a6f6b

Former-commit-id: 91d18750

* layer_norm_param_grad

* refine

* use tmp_buffer in fp16, because gpu_atomic_add is slow

* testcase

* refine

* refine

* refine

* fp16 test case

* fix

* rm trainable

* fix
Co-authored-by: Nguoran <guoran@oneflow.org>
Co-authored-by: NJuncheng <liujuncheng1022@gmail.com>
Former-commit-id: 5f5a128c

* support oneflow_api_registry

* fix code format

* Expand folder range

* optimize of_api_registry

* Update of_api_registry.h

add a blank line between constructor  and member function

* fix code format

* optimize oneflow_pybind_api

* fix code foemat

* fix oneflow.cmake

* add a blank line
Co-authored-by: Noneflow-bot <69100618+oneflow-bot@users.noreply.github.com>
Former-commit-id: 757d2eab

Former-commit-id: 9d1f7e586af02fe0f1f9aebe963921324b25100e

Merge branch 'dev_integrate_cfg' of https://github.com/Oneflow-Inc/oneflow into dev_replace_ErrorProto_to_cfgErrorProto


Former-commit-id: 3217d94b330c97973bf2928774b0f1d4df139374

Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into dev_replace_ErrorProto_to_cfgErrorProto


Former-commit-id: 77e9fa397f25f9e2df9782ddefb47a670d2b20b2

Former-commit-id: 2003a12531f6182c63c3956578c628cb1bcf8231

Former-commit-id: 3cb7fddc489ad4fabab0e0c49c24131ebdd4e64d

Former-commit-id: 2dd6ab82a4438f9cbcc5dca5b781c40958d8c89a

Former-commit-id: ef593e38

Former-commit-id: 7335207f

Former-commit-id: 29bade075df370fb2ca679a8da0d72cd5ef4e962

Merge branch 'dev_integrate_cfg' of https://github.com/Oneflow-Inc/oneflow into dev_replace_ErrorProto_to_cfgErrorProto

Conflicts:
	cmake/oneflow.cmake

Former-commit-id: 11cd3221802a2a46cfb3f5de8a8f6f84c4e8403f

Former-commit-id: aae9c767cd509dbc7dc97e3470381ae6da0a4e1e

Former-commit-id: da5eb816c0d8b214cbb9c2d81013e4d93c6aa74f

Former-commit-id: d8e330f282d8c4feb7149ec41dc747403f9c3452

Former-commit-id: ec7b4fb5f929fb303ee8bdcf393f8e33b933ccfe