* fix compile bug 1

* fix bug and add col num in ops

* fix compile bug and add scripts for rnn test

* fix for review

* test boxing op


Former-commit-id: b586114b

* add default operations to col_num field in forward and backward

* change some function in terms of reusing

* change name and test runtime

* rename some variables and restate some functions

* change name of CopyFieldFromInToOut()


Former-commit-id: c26a32b1

* updt names in ip

* updt comment && fix bug

* fix bug && updt proto

* rename innerproduct to fullyconnected

* updt field names in fully connected op

* roll back impl of initialization of bias

* fix bug of bias_multiplier_initializer

* remove redundant SetHasDataIdField

* remove redundant set_data_type

* refine code


Former-commit-id: 534138dc

Former-commit-id: 79d00a28

Former-commit-id: 8744e804

Former-commit-id: bf84fa09

* add l1 l2 regularization

* fix some problem

* Fix bug

* fix op API

* Fix a bug in op

* add l1l2

* remove regularization_diff_blob

* remove sign in kernel_util

* fix naming problem and add l1l2 function

* simplify code

* remove else

* update modelupdate API

* Fix bug

* ~

* fix bug

* Add BlobGemm function in kernel_util


Former-commit-id: bf91433c

Former-commit-id: 8ac8cf50

* add l1 l2 regularization

* fix some problem

* Fix bug

* fix op API

* Fix a bug in op

* add l1l2

* remove regularization_diff_blob

* remove sign in kernel_util

* fix naming problem and add l1l2 function

* simplify code

* remove else

* update modelupdate API

* Fix bug

* ~

* fix bug

* recurrent op


Former-commit-id: 9dd01051

* implement ForEachCurReadableRegst in actors

* fix -1 regst_desc_id in comm_net_actor

* fix bug

* change if order

* new

* fix null info in actor.cpp:ActUntilFail

* commit

* implement ForEachCurReadableRegst in actors

* commit

* change [] to at

* add const


Former-commit-id: 1917cb49

Former-commit-id: 323e956e

* imp TODO in chain node

* fix check


Former-commit-id: d6bc16bb

* add unexpectedRun in fw comp task for recurrent edge

* fix format


Former-commit-id: bc722f50

Former-commit-id: b1021869

Former-commit-id: 8fe2ddbd

Former-commit-id: 62966b62

* boxing for both cnn & rnn

* add ColIdOrder

* order uncertain

* fix boxing

* last -- max

* boxing kernel context

* fix int32

* new delete previous_pid_cid_

* nullptr after delete

* fix bug

* judge when max_col_num > 1


Former-commit-id: a260b163

Former-commit-id: 9094e8b9

Former-commit-id: c88b0569

Former-commit-id: 71fa9156

Former-commit-id: 2b63baae

Former-commit-id: 8f7dfd67

* fix bug & refine code

* fix bug

* remove ht in bindoutdiffregst func

* fix bugs

* recover build from bind

* fix regst num in ht_1_diff_regst

* fix regst num for rec_out

* replace ht_diff with rec_out_diff, replace ht_1_diff with rec_in_diff

* fix bugs && updt impl

* remove useless UNEXPECTED_RUN

* fix format

* fix bugs

* add interface GetConsumedRegstWrapper && refine code

* rollback changes of VirtualConsumeInRegst func


Former-commit-id: a0343401

Former-commit-id: 1f7765e0

Former-commit-id: 271de42e

* change dataloader to move col_id and max_col_id to register

* some miner changes


Former-commit-id: 2e47a8ee

Former-commit-id: f27ace65

* fw task node for recurrent out edge

* recurrent task node 2 out regst

* set min regst num of rec_ht_regst

* todo is recurrent out edge

* refine name

* implement is recurrent out edge for comp task node

* reduce code for last impl

* fix for sub class fuc

* change bind out regst in fw task node by SunPeiWen

* fix for review

* unexpected_run for virtual func

* fix bug by zyeric fault

* change interface: delete isrecurrentoutedge add get succ/pred chain node on edge

* rec_ht

* fix name

* fix name

* refine fw(recurrent/nomal) compute task node

* fix bug of bind in regst

* delete node

* fix for review

* abstract new func for succ/pred chain node on edge

* fix max regst in model parallel

* fix back


Former-commit-id: 5970b3d3

* feat: update accumulate actor

* fix: fix comment and add enum class ColIdOrder

* refactor: rename ColIdOrder::UnSet to ColIdOrder::UnCertain

* chore: rename IsFirstRegstInPieceOfThisOrder and move it out of Actor Class

* fix: initialize col_id/max_col_id of blobheader as 0 instead of -1


Former-commit-id: 2e100ea6

Former-commit-id: 65258e6c

Former-commit-id: cad45673

* refine recurrent op

* ht -> rec_ht , out-> ht

* notes

* refine recurretn op conf

* sigmoid

* for blob header and col num


Former-commit-id: 988f5d9b

Former-commit-id: 2fcd8431

Former-commit-id: cf20a9f8

Former-commit-id: 29d76763

* change dataloader

* make basic_data_loader_kernel.cpp readable

* debug

* debug dataloader

* change something due to review

* do some changes

* refactor to reduce code

* remove comment

* refactor the dataloader kernel

* debug

* debug a runtime bug

* remove fillblobwithzero method

* changes based on reviews

* debug and make it runnable

* change the declarations due to zyeric's change request

* changes due to willzhang's change request

* make it able to compile

* change the type of blob header and move the position of DataLoadStatus

* change the declaration of function ReadColFromBufferToOutBlob()

* change the wrong use of memcpy() and memset()

* add 0 after the end of each line

* refactor ReadOneDataContent() function

* revert a wrong change

* remove unnecessary CHECK()

* rename and move some declaration

* bug for compiles


Former-commit-id: d51a32e1

Former-commit-id: bef885fb

Former-commit-id: b6e0e3ad

Former-commit-id: c6789e54

* add l1 l2 regularization

* fix some problem

* Fix bug

* fix op API

* Fix a bug in op

* add l1l2

* remove regularization_diff_blob

* remove sign in kernel_util

* fix naming problem and add l1l2 function

* simplify code

* remove else

* update modelupdate API

* Fix bug

* ~

* fix bug


Former-commit-id: 8815ce16