Skip to content

O
oneflow

Oneflow-Inc / oneflow
上一次同步 2 年多

通知 13
Star 2733
Fork 0
O
oneflow

前往新版Gitcode，体验更适合开发者的 AI 搜索 >>

切换分支/标签
  1. 26 10月, 2021 1 次提交
  3. 15 10月, 2021 1 次提交
  5. 01 10月, 2021 1 次提交
    • L
      Refactor eager critical section (#6408) · 092fef2a
      由 Li Xinqi 提交于 
      * backup code

* EventRecord

* auto format by CI

* remove deprecated binary test cases

* refactor valatile to atomic

* add StreamType::InitInstructionStatusIf/StreamType::DeleteInstructionStatusIf

* merge from branch profiling_nn_graph

* address comments

* EventRecordProvider

* more comments for XXXStatusQuerier::SetLaunched

* more comments for SharedEventRecord::Init

* address comments
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Ncheng cheng <472491134@qq.com>
      092fef2a
  7. 30 9月, 2021 2 次提交
  9. 24 9月, 2021 2 次提交
  11. 14 9月, 2021 1 次提交
  13. 13 9月, 2021 1 次提交
  15. 12 9月, 2021 1 次提交
  17. 10 9月, 2021 1 次提交
  19. 09 9月, 2021 1 次提交
  21. 08 9月, 2021 1 次提交
  23. 03 9月, 2021 1 次提交
  25. 01 9月, 2021 1 次提交
  27. 31 8月, 2021 1 次提交
  29. 27 8月, 2021 1 次提交
  31. 21 8月, 2021 1 次提交
  33. 17 8月, 2021 2 次提交
  35. 11 8月, 2021 1 次提交
  37. 10 8月, 2021 1 次提交
    • L
      Get parallel_id and parallel_num through rank and world size in DDP (#5717) · 963cc01b
      由 leaves-zwx 提交于 
      * get parallel_id and parallel_num through rank and world size in dpp

* address review

* coco reader support parallel distribution

* fix that device not set

* test parallel for ofrecord reader

* update test

* update test

* erase illegal check

* test success

* add GPTIndexedBinDataReader module

* test distributed GPTIndexedBinDataReader

* fix that LogicalTensorDesc4ArgNameAndIndex unimplemented in local kernel init context

* graph handle TensorTuple output

* fix COCOReader forward

* update test_gpt_data_loader

* test_coco_reader

* auto format by CI

* fix block call return not supporting tensor tuple

* fix IsMirroredParallelContext

* convert TensorTuple to tuple of Tensor

* check
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
      963cc01b
  39. 08 8月, 2021 1 次提交
  41. 04 8月, 2021 1 次提交
    • X
      nn.Graph with training (#5662) · c39d083d
      由 Xiaoyu Xu 提交于 
      * pass test on linear with training

* Refactor RuntimeCtx for multi-runtime

* refactor inplace to support nn graph

* block support iterator

* block iter add check

* fix scalar_mul op conf build

* deal with inplace after merge master

* add alexnet graph test

* add cpu test and format

* cout to glog

* deal with Job run finish bug

* refactor lazy deal with inplace

* deal with 0D tensor

* update data path

* address review

* deal with lazy default attr

* mv according to ci

* merge master

* fix for ci

* fix for ci limit
Co-authored-by: Nchengtbf <472491134@qq.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
      c39d083d
  43. 28 7月, 2021 2 次提交
  45. 26 7月, 2021 1 次提交
    • S
      Rm oneflow export and experimental (#5589) · 4780aae5
      由 Shenghang Tsai 提交于 
      * refine

* change abs.py for test

* refine

* refine

* refine

* refine

* refine

* refine

* add

* refeom

* refine

* refine

* refien

* refine

* refine

* refine

* refien

* add check

* refine

* refine

* refine

* skip license

* refine

* refien

* refine

* refien

* refien

* refine

* refine

* refine acos

* refien

* remove enable_eager_execution() in oneflow/python

remove @experimental_api decorator for apis, except that ones
having same name with single client api.

* remove all import oneflow.experimental in doctest

* replace oneflow.experimental with oneflow in docstr

* reformat

* auto format by CI

* refine

* refien

* refine

* refien

* refine

* refine

* refine

* refein

* refine

* remove signle client twice api

* refine

* refine

* refine

* refien

* refien

* refine

* refien

* refien

* refien

* refine

* refein

* refein

* refein

* refeine

* refein

* refien

* fix

* remove experimental of nn

* refine

* rm oneflow.experimental.F

* refein

* refine

* refien

* refein

* okokok

* remove experimental in test/modules and test/tensor

* refein

* oneflow.python => oneflow

* rm absolute_import

* remove experimental in test/dataloader

* remove experimental call in nn/modules

* remove experimental from nn/ framework/ test_utils

* remove experimental in functionalpy

* refein

* refein

* refein

* refein

* fix python reserved words

* refein

* fix alias

* refein

* refien

* refein

* refein

* refein

* refein

* refine

* refein

* refein

* refein

* add

* refein

* refein

* remove experimental of new merge

* fix

* rm

* refien

* mv setuppy

* refien

* skip fmod grad test

* remove experimental of fmod

* refein

* setup py proj

* refien

* fix error when do multi case test on graph

* remove outdated rst files

* refine and add new rst files

* refein

* check in src

* format

* revert test_fmod.py

* refien

* remove all experimental in python/nn dir (done!)

* remove experimental in test/graph

* refein

* refein

* format

* add ignore

* refein

* add linagl

* refei

* refine rst

* update build whl

* fix

* refein

* refein

* refine

* refine

* auto format by CI

* add license

* add step

* refine

* refien

* refein

* update paths

* updatte paths

* refine

* reifne

* refein

* refein

* refein

* rrefien

* refien

* refein

* refine scatter_nd

* refine oneflow.rst

* refein

* from . import linalg
from . import autograd
from . import optim

* refein

* refine load_minst's docstring

add image.rst module.rst

refine rst files

* reformat

* remove Graph api

* fix file name

* refien

* fix oneflow.nn.image

* import oneflow.nn.image

* refein

* fix from . import lr_scheduler

* fix

* refeine

* refein

* refein

* rm todo

* rm unused box

* refein

* refien

* refein

* refien

* add ignore

* fix distributed api

* refein

* refein

* refine

* refine

* fix doctest

* refein

* update

* refien

* refein

* refein

* refein

* refein

* refein

* fix colition

* refien

* refien

* fix doc

* fix module doc

* fmt

* auto format by CI

* create moduels

* fix

* fix

* auto format by CI

* fix proto problem

* update cmake

* refien

* refein

* refienrefein

* refien

* refein

* add ignore

* refein

* refein

* refien

* auto format by CI

* refein

* refei

* refein

* refien

* refein

* udpate commit

* add ignore

* fix

* refein

* refein

* fix saved_model

* rjefein

* refein
Co-authored-by: NGehangZhang <649975329@qq.com>
Co-authored-by: NYaoChi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Nstrint <xiaoyulink@gmail.com>
      4780aae5
  47. 25 7月, 2021 1 次提交
    • C
      nn.Graph call and launch impl (#5580) · 3dcb9939
      由 cheng cheng 提交于 
      * nn.Graph call and launch impl

* Fix bug of 1.pybind11 list->vector; 2.GlobalJobDesc in NNGraph Compile

* Fix bug: AutoTick Multi-Client callback notifier lbn

* Fix bug: callback notifier op check in multi-client

* Fix bug: CheckOpGraph in multi-client

* Fix bug of MultiClientSessionContext new ProfileConf

* Fix bug of create NONE type kernel; add note; refine PlanUtil::PopulateOpAttibute

* Fix bug: CallbackNotifier kernel dtype with uint8; change tick dtype; Feat: add Kernel create fault-tolerant

* inputs tensor list

* rollback of hack

* Using TensorTuple for RunLazyNNGraph

* Fix bug: WaitAndSendIdsKernel handle for Multi-Client

* skip graph runnable test

* refine

* skip test graph optimizer for nn.Graph train job completer error

* Skip SetDefaultVariableConf JobPass in Multi-Client

* Fix bug: register_variable_op_names_and_tensors args with TensorTuple

* refine test graphs

* rm experimental in new scripts
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
      3dcb9939
  49. 24 7月, 2021 1 次提交
    • G
      remove outdated doctest api and "@experimental_api" (#5564) · 8e9e61db
      由 GehangZhang 提交于 
      * change doctest code in abs.py

* del >>> flow.enable_eager_execution() in  modules/abs.py

* del @experimental_api

* del @experimental_api

* slove doctest not pass

* change abs.py for test

* refine acos

* remove enable_eager_execution() in oneflow/python

remove @experimental_api decorator for apis, except that ones
having same name with single client api.

* remove all import oneflow.experimental in doctest

* replace oneflow.experimental with oneflow in docstr

* reformat

* auto format by CI

* remove signle client twice api

* refine

* remove experimental of nn

* okokok

* remove experimental in test/modules and test/tensor

* remove experimental in test/dataloader

* remove experimental call in nn/modules

* remove experimental from nn/ framework/ test_utils

* remove experimental in functionalpy

* remove experimental of new merge

* skip fmod grad test

* remove experimental of fmod

* fix error when do multi case test on graph

* remove outdated rst files

* refine and add new rst files

* format

* revert test_fmod.py

* remove all experimental in python/nn dir (done!)

* remove experimental in test/graph

* add linagl

* refine rst

* refine scatter_nd

* refine oneflow.rst

* refine load_minst's docstring

add image.rst module.rst

refine rst files

* reformat

* remove Graph api

* fix file name

* fix index
Co-authored-by: NYaoChi <later@usopp.net>
Co-authored-by: Noneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Nstrint <xiaoyulink@gmail.com>
Co-authored-by: Ntsai <jackalcooper@gmail.com>
      8e9e61db
  51. 22 7月, 2021 1 次提交
    • X
      Fea/nn graph/optimizer (#5533) · e5589c0a
      由 Xiaoyu Xu 提交于 
      * add test on add input to graph

* add var into graph

* LazyInterpreter for FetchOutputOpExpr and set op parallel_distribution

* refine input var build

* split file

* rename

* mini refine

* add complete_graph_config

* add sgd

* fix style

* Add note

* LazyInterpret::ApplyImpl for UserOpExpr

* refine test scripts

* add output to graph

* format

* tensor.backward to lazy_add_loss(tensor)

* Fix bug of LazyInterpret UserOpExpr for change output lbns

* Add test user op expr test

* fix note mistake

* add userop and test

* rename

* add idea on l2 to variable op

* add scale in sgd

* refactor add loss

* revert change on tensor_name_scope

* fix typo

* check is lazy in tensor.backward

* rm useless state_tensortuple in nn.Graph
Co-authored-by: Nchengtbf <472491134@qq.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
      e5589c0a
  53. 21 7月, 2021 1 次提交
    • X
      Fea/nn graph/forward graph (#5516) · ab8aab8b
      由 Xiaoyu Xu 提交于 
      * add test on add input to graph

* add var into graph

* LazyInterpreter for FetchOutputOpExpr and set op parallel_distribution

* refine input var build

* split file

* rename

* mini refine

* Add note

* LazyInterpret::ApplyImpl for UserOpExpr

* refine test scripts

* add output to graph

* format

* Fix bug of LazyInterpret UserOpExpr for change output lbns

* Add test user op expr test

* fix note mistake

* add userop and test

* address review

* address review

* save i/o/s op_name and tensor for c_nn_graph

* address review

* adjust test

* refine build_graph_state
Co-authored-by: Nchengtbf <472491134@qq.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
      ab8aab8b
  55. 15 7月, 2021 1 次提交
  57. 13 7月, 2021 1 次提交
    • X
      Fea/nn graph/graph build ctx (#5420) · 67272d28
      由 Xiaoyu Xu 提交于 
      * graph api

* add graph dummy test

* add test

* add recursive module mode

* graph.build test pass

* add detail check on graph inner node

* support config and train

* add repr for debug

* test buffer

* test buffer add

* refine test

* add comment

* refine test

* refactor Node to Block

* add named_state

* refine Graph.named_state()

* add state_tensortuple

* graph._compile()

* add mc session 0

* nn.graph: state tuple to private var; add BlockType; add simple multi client session

* NNGraphIf

* rm old graph.cpp

* nn.graph: add cpp NNGraph; export and call NNGraph

* add comment

* nn.Graph: rm prototype MultiClientSession

* nn.Graph: rm prototype MultiClientSession test

* nn.Graph: add TODO

* nn.Graph: format for review

* nn.Graph: format

* nn.Graph: format

* nn.Graph: pass flake8 check

* job_build_ctx

* support lazy context

* format

* lazy mode

* format

* format

* lazy mode add test

* debug session

* init session and job_build_context

* rm temp code

* build default scope

* add default scope

* add scope proto for debug

* chech scope

* format

* refine MultiClientSession.resource

* address review

* lazy init scope stack in single-client, instantly init scope stack after MultiClientSession created in multi-client

* fix typo

* address review

* fix clear default session

* merger and test
Co-authored-by: NXinqi Li <lixinqi0703106@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Nleaves-zwx <kunta0932@gmail.com>
Co-authored-by: Ncheng cheng <472491134@qq.com>
      67272d28
  59. 09 7月, 2021 2 次提交
    • X
      Fea/nn graph/graph name (#5413) · 041e441f
      由 Xiaoyu Xu 提交于 
      * graph api

* add graph dummy test

* add test

* add recursive module mode

* graph.build test pass

* add detail check on graph inner node

* support config and train

* add repr for debug

* test buffer

* test buffer add

* refine test

* add comment

* refine test

* refactor Node to Block

* add named_state

* refine Graph.named_state()

* add state_tensortuple

* graph._compile()

* add mc session 0

* nn.graph: state tuple to private var; add BlockType; add simple multi client session

* NNGraphIf

* rm old graph.cpp

* nn.graph: add cpp NNGraph; export and call NNGraph

* add comment

* nn.Graph: rm prototype MultiClientSession

* nn.Graph: rm prototype MultiClientSession test

* nn.Graph: add TODO

* nn.Graph: hack to get Graph object name

* nn.Graph: get obj name

* nn.Graph: get obj name 2

* nn.Graph: format for review

* nn.Graph: format

* nn.Graph: format

* nn.Graph: pass flake8 check

* Update graph.py

* name with init count

* name with init count 2
Co-authored-by: NXinqi Li <lixinqi0703106@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
      041e441f
    • X
      rm nn.Graph.train (#5424) · a94748f1
      由 Xiaoyu Xu 提交于 
      * rm nn.Graph.train

* Update graph.py

GraphCofig default predict

* Update test_graph.py
      a94748f1
  61. 07 7月, 2021 1 次提交
    • X
      nn.Graph python (#5309) · 0a821433
      由 Xiaoyu Xu 提交于 
      * graph api

* add graph dummy test

* add test

* add recursive module mode

* graph.build test pass

* add detail check on graph inner node

* support config and train

* add repr for debug

* test buffer

* test buffer add

* refine test

* add comment

* refine test

* refactor Node to Block

* add named_state

* refine Graph.named_state()

* add state_tensortuple

* graph._compile()

* add mc session 0

* nn.graph: state tuple to private var; add BlockType; add simple multi client session

* NNGraphIf

* rm old graph.cpp

* nn.graph: add cpp NNGraph; export and call NNGraph

* add comment

* nn.Graph: rm prototype MultiClientSession

* nn.Graph: rm prototype MultiClientSession test

* nn.Graph: add TODO

* nn.Graph: format for review

* nn.Graph: format

* nn.Graph: format

* nn.Graph: pass flake8 check
Co-authored-by: NXinqi Li <lixinqi0703106@163.com>
Co-authored-by: Noneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Ncheng cheng <472491134@qq.com>
      0a821433