* branch init

* logical graph

* chain graph with bp struct

* backward done

* loss record struct

* build model struct

* compute task node

* fix bugf

* GenSortedCompTaskNodes

* clone op with boxing thread

* BuildSubTaskGraphMethod

* remove addclone for data_loader

* copy task node

* task graph build done

* fix boxing

* add TODO

* update

* update

* MaxModelSplitNum

* data part num

* DataConcatAndDataSplit


Former-commit-id: 8ed98d5c

Former-commit-id: 7b73c17e

* conv pooling kernel

* fix name

* pooling and convolution kernel register

* model udpate kernel register

* fix convolutionOp and poolingOp

* LogicalBlob -> string

* remove comment

* dev_opkernel init

* data loader refine

* acc kernel

* refine kernel util

* inner product

* register template kernel creator

* IsInt and IsFloat

* refine loss op (#290)

* refine loss op

* fix for review

* ibn2lbn obn2lbn

* pooling

* updt relu operator (#291)

* updt relu operator

* update relu and model_save operator

* add check between input blob desc and op config for relu operator

* record

* fix typo

* copy hd

* copy comm net

* softmax

* update shape count

* update interface init from op_conf

* loss record

* Update model_save and relu kernel (#295)

* updt relu operator

* update relu and model_save operator

* add check between input blob desc and op config for relu operator

* update model_save and relu kernel

* refine model_save and relu kernel

* concat

* Convolution op and kernel (#298)

* fix bug: error device type for boxing

* fix bug: error device type for copy_hd

* add convolution_op

* op_conf

* using MACRO_PAIR

* fix coding style

* rollback

* fix convlution op

* mu loss kernel (#294)

* mu loss kernel

* softmax_loss_kernel

* refine softmax op kernel

* fix macro

* fix bug of kernel manager

* fix compile bug, function_cast

* remove const

* loss record

* acc

* refine kernel test common

* refine kernel util

* ktcommon

* preprocess macro utils of sequence

* remove MACRO_PAIR

* update macro

* rmsprop

* rmsprop infer blob desc

* normal model update

* clone kernrl

* Dev opkernel jueliu 1 (#302)

* dummy

* reverted commits from master

* momentum model update op and kernel (#301)

* momentum

* head file

* fix some problem

* momentum op

* fix bug

* fix bug

* Dev lixinqi pp cartesian product (#304)

* refine pp_seq. add test code

* preprocessor sequence product: (0)(1) * (0)(1) -> ((0, 0))((1, 0))((0, 1))((1, 1))

* Dev lixinqi pp msvc compliance (#310)

* fix macro compliant bug on windows

* double expandation when deal with #define APPLAY(macro, tuple) macro tuple

* Dev lixinqi refactor preprocessor (#312)

* merge pp_*.h to pp_internal.h. not done yet

* fix macro compliant bug on windows

* double expandation when deal with #define APPLAY(macro, tuple) macro tuple

* set limit of seq size 64

* clone op test

* clone op test

* rename

* update include guard

* first arg

* rename macro

* remove

* remove for_each_pair

* remove

* clone kernel

* define BOOL_SEQ

* concat op test

* opkernel mdupdatetest (#306)

* momentum

* head file

* fix some problem

* momentum op

* normal model update

* using MICRO

* normal model update using new MICRO

* fix name problem

* OF_PP_FIRST_ARG -> OF_PP_PAIR_FIRST; OF_PP_SECOND_ARG -> OF_PP_PAIR_SECOND (#315)

* fix compile bug

* concat kernel test

* correctly handle __VA_ARGS__ related macros in msvc (#319)

* Merge function in rmsprop model update (#318)

* momentum

* head file

* fix some problem

* momentum op

* rmsprop

* Rmsprop and momentum model update kernel test (#316)

* momentum

* head file

* fix some problem

* momentum op

* normal model update

* using MICRO

* normal model update using new MICRO

* fix name problem

* rmsprop model update

* Add momentum and rmsprop model update kernel test

* copy hd

* rewrite accumulate_kernel_test.cpp (#308)

* update accumulate_kernel_test.cpp

* pass accumulate kernel test

* KTCommon -> KTC

* apply OF_PP_SEQ_PRODUCT_FOR_EACH_TUPLE

* DeviceTypeSeq -> DEVICE_TYPE_SEQ

* FIRST_ARG -> PAIR_FIRST()

* BuildBnInOp2BlobPtr -> BuildBnInOp2BlobMap

* bn2blob_ptr -> bn2blob

* construct blob_desc in one line
build bn2blob item in one line
change some var names
move BuildAccumulateKernel function to top

* cnn kernel test

* update code owner

* Dev opkernel cc (#314)

* fix macro compliant bug on windows

* int32 to int

* ip kernel test

* fix for review

* fix for review2

* delete array define

* fix ip kernel test

* rm data loader

* refine relu kernel test (#305)

* refine relu kernel test

* add data id test for relu kernel

* update SetDataId and relu_kernel_test

* refine relu_kernel_test

* add sequence generating macros

* refine boxing_kernel_test

* remove function SetDataId and update boxing kernel test

* cnn op test

* fix bug in vs narrowing conversion when init.list (#322)

* refine softmax kernel test (#323)

* refine softmax loss kernel for two template data type (#321)

* refine softmax loss kernel for two template data type

* typename PredType

* multi log loss kernel refine (#324)

* rewrite innerproduct op test (#313)

* update accumulate_kernel_test.cpp

* pass accumulate kernel test

* rewrite innerproduct_op_test

* bakup before delete old codes

* rewrite innerproduct_op_test.cpp

* apply OF_PP_SEQ_PRODUCT_FOR_EACH_TUPLE

* move PARALLEL_POLICY_SEQ to util.h

* construct BlobDesc onetime

* CHECK BlobDesc ==

* FIRST_ARG -> PAIR_FIRST()

* CHECK -> ASSERT_EQ
op->model_bns().at(0) -> weight, same as bias and bias_multiplier
replace one-time variable
bn2blobdesc_ptr -> bn2blobdesc

* remove one-time variable-blob_desc

* 40 -> out_num

* remove usless #include
replace fp -> bn2blob_desc_func
use "in", "out", "weight"...

* ASSERT_EQ -> ASSERT_TRUE

* PoolingKernelTest (#325)

* change data format

* random

* softmax loss kernel test (#327)

* softmax loss op test (#326)

* softmax loss op test

* fix bug in softmaxloss op test

* refine relu_op_test, softmax_op_test and boxing_op_test (#328)

* PoolingOpTest (#329)

* Poolingkerneltest micro (#330)

* PoolingOpTest

* Change PoolingKernelTest Micro

* Dev multinomial logistic loss (#331)

* PoolingOpTest

* PoolingOpTest

* Change PoolingKernelTest Micro

* MultinomialLogisticLossKernelTest

* add KTC Micro

* Mll op test and fix a bug (#332)

* PoolingOpTest

* PoolingOpTest

* Change PoolingKernelTest Micro

* mll op test and fix a bug

* fix include

* model parallel for conv

* change paramters to '_h & _w' in pooling && conv op/kernel

* fix comments

* fix proto

* fix bias_term bugs in conv && ip operator

* fix backward process in case the in_diff is nullptr --- e.x. previous layer is data-loarder.

* fix space format

* fix bias_multiplier

In proto_io, 4 methods added for type casting.

dag -> graph