* branch init

* logical graph

* chain graph with bp struct

* backward done

* loss record struct

* build model struct

* compute task node

* fix bugf

* GenSortedCompTaskNodes

* clone op with boxing thread

* BuildSubTaskGraphMethod

* remove addclone for data_loader

* copy task node

* task graph build done

* fix boxing

* add TODO

* update

* update

* MaxModelSplitNum

* data part num

* DataConcatAndDataSplit


Former-commit-id: 8ed98d5c

* fix label bug

* max_data_id_length

* no problem with dataloader kernel

* remove modify

* CHECK max_data_id

* data_id

* fix train problem

* fix some problem for data id

* Unified naming

* clang-format

* predict_job.proto

* concatKernel add data_id

* change \n position and fix some problem in kernel

* seperate copy data and copy dataid

* add paras in concat kernel

* solve record output problem

* remove changes in train_placement

* rename CopyDataIdFromIbToAllOb API and dataloader mathod

* unified naming

* unify naming

* unify naming

* unify naming

* change CopyDataIdFromSoleIbToAllObIfNeed API

* Fix CHECK range bug in concat.cpp


Former-commit-id: 6db713e5

* add kernel regst for all other kernel

* fix bug for more replace in softmaxlosskernel test

* creators

* remove datatype in op_conf except data loader

* fix for ipkernel.cpp

* dev_opkernel init

* data loader refine

* acc kernel

* refine kernel util

* inner product

* register template kernel creator

* IsInt and IsFloat

* refine loss op (#290)

* refine loss op

* fix for review

* ibn2lbn obn2lbn

* pooling

* updt relu operator (#291)

* updt relu operator

* update relu and model_save operator

* add check between input blob desc and op config for relu operator

* record

* fix typo

* copy hd

* copy comm net

* softmax

* update shape count

* update interface init from op_conf

* loss record

* Update model_save and relu kernel (#295)

* updt relu operator

* update relu and model_save operator

* add check between input blob desc and op config for relu operator

* update model_save and relu kernel

* refine model_save and relu kernel

* concat

* Convolution op and kernel (#298)

* fix bug: error device type for boxing

* fix bug: error device type for copy_hd

* add convolution_op

* op_conf

* using MACRO_PAIR

* fix coding style

* rollback

* fix convlution op

* mu loss kernel (#294)

* mu loss kernel

* softmax_loss_kernel

* refine softmax op kernel

* fix macro

* fix bug of kernel manager

* fix compile bug, function_cast

* remove const

* loss record

* acc

* refine kernel test common

* refine kernel util

* ktcommon

* preprocess macro utils of sequence

* remove MACRO_PAIR

* update macro

* rmsprop

* rmsprop infer blob desc

* normal model update

* clone kernrl

* Dev opkernel jueliu 1 (#302)

* dummy

* reverted commits from master

* momentum model update op and kernel (#301)

* momentum

* head file

* fix some problem

* momentum op

* fix bug

* fix bug

* Dev lixinqi pp cartesian product (#304)

* refine pp_seq. add test code

* preprocessor sequence product: (0)(1) * (0)(1) -> ((0, 0))((1, 0))((0, 1))((1, 1))

* Dev lixinqi pp msvc compliance (#310)

* fix macro compliant bug on windows

* double expandation when deal with #define APPLAY(macro, tuple) macro tuple

* Dev lixinqi refactor preprocessor (#312)

* merge pp_*.h to pp_internal.h. not done yet

* fix macro compliant bug on windows

* double expandation when deal with #define APPLAY(macro, tuple) macro tuple

* set limit of seq size 64

* clone op test

* clone op test

* rename

* update include guard

* first arg

* rename macro

* remove

* remove for_each_pair

* remove

* clone kernel

* define BOOL_SEQ

* concat op test

* opkernel mdupdatetest (#306)

* momentum

* head file

* fix some problem

* momentum op

* normal model update

* using MICRO

* normal model update using new MICRO

* fix name problem

* OF_PP_FIRST_ARG -> OF_PP_PAIR_FIRST; OF_PP_SECOND_ARG -> OF_PP_PAIR_SECOND (#315)

* fix compile bug

* concat kernel test

* correctly handle __VA_ARGS__ related macros in msvc (#319)

* Merge function in rmsprop model update (#318)

* momentum

* head file

* fix some problem

* momentum op

* rmsprop

* Rmsprop and momentum model update kernel test (#316)

* momentum

* head file

* fix some problem

* momentum op

* normal model update

* using MICRO

* normal model update using new MICRO

* fix name problem

* rmsprop model update

* Add momentum and rmsprop model update kernel test

* copy hd

* rewrite accumulate_kernel_test.cpp (#308)

* update accumulate_kernel_test.cpp

* pass accumulate kernel test

* KTCommon -> KTC

* apply OF_PP_SEQ_PRODUCT_FOR_EACH_TUPLE

* DeviceTypeSeq -> DEVICE_TYPE_SEQ

* FIRST_ARG -> PAIR_FIRST()

* BuildBnInOp2BlobPtr -> BuildBnInOp2BlobMap

* bn2blob_ptr -> bn2blob

* construct blob_desc in one line
build bn2blob item in one line
change some var names
move BuildAccumulateKernel function to top

* cnn kernel test

* update code owner

* Dev opkernel cc (#314)

* fix macro compliant bug on windows

* int32 to int

* ip kernel test

* fix for review

* fix for review2

* delete array define

* fix ip kernel test

* rm data loader

* refine relu kernel test (#305)

* refine relu kernel test

* add data id test for relu kernel

* update SetDataId and relu_kernel_test

* refine relu_kernel_test

* add sequence generating macros

* refine boxing_kernel_test

* remove function SetDataId and update boxing kernel test

* cnn op test

* fix bug in vs narrowing conversion when init.list (#322)

* refine softmax kernel test (#323)

* refine softmax loss kernel for two template data type (#321)

* refine softmax loss kernel for two template data type

* typename PredType

* multi log loss kernel refine (#324)

* rewrite innerproduct op test (#313)

* update accumulate_kernel_test.cpp

* pass accumulate kernel test

* rewrite innerproduct_op_test

* bakup before delete old codes

* rewrite innerproduct_op_test.cpp

* apply OF_PP_SEQ_PRODUCT_FOR_EACH_TUPLE

* move PARALLEL_POLICY_SEQ to util.h

* construct BlobDesc onetime

* CHECK BlobDesc ==

* FIRST_ARG -> PAIR_FIRST()

* CHECK -> ASSERT_EQ
op->model_bns().at(0) -> weight, same as bias and bias_multiplier
replace one-time variable
bn2blobdesc_ptr -> bn2blobdesc

* remove one-time variable-blob_desc

* 40 -> out_num

* remove usless #include
replace fp -> bn2blob_desc_func
use "in", "out", "weight"...

* ASSERT_EQ -> ASSERT_TRUE

* PoolingKernelTest (#325)

* change data format

* random

* softmax loss kernel test (#327)

* softmax loss op test (#326)

* softmax loss op test

* fix bug in softmaxloss op test

* refine relu_op_test, softmax_op_test and boxing_op_test (#328)

* PoolingOpTest (#329)

* Poolingkerneltest micro (#330)

* PoolingOpTest

* Change PoolingKernelTest Micro

* Dev multinomial logistic loss (#331)

* PoolingOpTest

* PoolingOpTest

* Change PoolingKernelTest Micro

* MultinomialLogisticLossKernelTest

* add KTC Micro

* Mll op test and fix a bug (#332)

* PoolingOpTest

* PoolingOpTest

* Change PoolingKernelTest Micro

* mll op test and fix a bug

* Split box axis is now fixed at 0

* add test cases for concat-split boxes

* add test for concat-axis at 0

* add unexpected_run in inferfwcopyrules

* initial draft for boxing kernel

* draft fix common boxing kernel design

* draft fix common boxing kernel design

* draft fix common boxing kernel design

* fix boxing_kernel.cu and some compiling errors

* fix some compiling errors, still have issues

* compile errors fixed, need refine && test

* fix issues of copy rules in clone box case

* Add simple tests, still debugging...

* fixed bug in InferCopyRules

* fixed issues with memcpy, passed manual test for concat-split boxes

* Add concat-split test case, issues fixed

* add test cases for concat-clone && add-clone boxes of cpu version

* fix typos

* fix types

* fix some issues identified by Will

* add function CopyDataFromRules

* fix add-split forward && concat-clone backward, add test case for add-split forward cpu version

* Modified boxing_kernel by KernelUtil, test cases not finished yet.

* Modified boxing_kernel by KernelUtil, test cases not finished yet.

* re-write by KernelUtil

* delete DeviceType::kALL

* delete DeviceType::kALL

* fix include files issue

* fixed issues found by Will

* fix issues

* fix floating_point_type to FloatingPointType

* fix issues, delete add-split box rules

* fix issues with comments

* change 'slice' to 'concat_dim'

* fix issues

* add NOTE for mutable vars

* fix test file issues

* fix formats

* add a test case with nullptr on concat-split box

* fix comments

* delete test cases

* change BnInOp2BlobPtr to BnInOp2Blob

* fix typos