Resolves: #5257 
Signed-off-by: Nfishpenguin <kun.yu@zilliz.com>

- set dd request's time tick on master
- send time tick into dd channel

See also: #5291

Signed-off-by: yefu.chen yefu.chen@zilliz.com

* Add SaveBinlogPaths grpc in dataservice

* fix merged compile error
Signed-off-by: NCongqi Xia <congqi.xia@zilliz.com>

- add cases for collection with data
- cancel import *

See also: #5302 #5304 

Signed-off-by: ThreadDao yufen.zong@zilliz.com

DDL won't auto-flush now

See also: #5289 , #5220 
Signed-off-by: Nyangxuan <xuan.yang@zilliz.com>

See also: #5253 
Signed-off-by: Nfishpenguin <kun.yu@zilliz.com>

Add SessionManager.
Resolves: #5174 
Signed-off-by: Ngodchen <qingxiang.chen@zilliz.com>

Signed-off-by: Nquicksilver <zhifeng.zhang@zilliz.com>

Resolves: #5293 

Restore segment flush stream & statistic stream to last success pos
Signed-off-by: NCongqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Nyangxuan <xuan.yang@zilliz.com>

- add cases for creating collections
- change unique str from var to function
- improve coding standards

See also: #5273 #5285 

Signed-off-by: ThreadDao yufen.zong@zilliz.com

Before this PR, DataNode considered auto-flush a valid flush
complete. It's wrong. So I open this PR to correct this behaviour
in DataNode.

Now binlog paths from auto-flush will be buffered in replica,
waiting until the manul flush to save into etcd all together.

See also: #5220, #5268
A follow up job of #5271 
Signed-off-by: Nyangxuan <xuan.yang@zilliz.com>

This document includes changes about dataservice and new features 
to support the recovery after it crashed.
Signed-off-by: Nsunby <bingyi.sun@zilliz.com>

Add exclusive parameter.
Issue #5174 
Signed-off-by: Ngodchen <qingxiang.chen@zilliz.com>

query base on timeline, add timestamp on these queries:
- ShowCollections
- HasCollection
- DescribeCollection

Resolves: #5219
Signed-off-by: Nyefu.chen <yefu.chen@zilliz.com>

Auto-flush of a segment is not considered flush-completed.
So we need to buffer binlog paths generated by auto-flush.

See also: #5268 #5220 
Signed-off-by: Nyangxuan <xuan.yang@zilliz.com>

let meta support snapshot
so collection could query meta with timestamp

Resolves: #5219
Signed-off-by: Nyefu.chen <yefu.chen@zilliz.com>

Signed-off-by: NXiangyu Wang <xiangyu.wang@zilliz.com>

Add session util.
Issue #5174 
Signed-off-by: Ngodchen <qingxiang.chen@zilliz.com>

Signed-off-by: Nyangxuan <xuan.yang@zilliz.com>

The design procedure can be seen: #5220 

Updata PhysicalChannelNames and VirtualChannelNames based
on ShardsNum when create collection, and update collVChan2Chan
map.

Resolves: #5240 
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

- update get_connection to create_connection
- add case level for collection
- optimize collection cases


Signed-off-by: ThreadDao yufen.zong@zilliz.com

* Change volumes with pytest docker-compose
Signed-off-by: Nquicksilver <zhifeng.zhang@zilliz.com>

* Fix run pytest bug
Signed-off-by: Nquicksilver <zhifeng.zhang@zilliz.com>

Signed-off-by: Nquicksilver <zhifeng.zhang@zilliz.com>

Add a part of the test case for the Collection interface

See also: #5224 #5231 

Signed-off-by: ThreadDao yufen.zong@zilliz.com

Remove CreateIndexTask struct and indexTaskQueue

Resolves: #5228 
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

Signed-off-by: NXiangyu Wang <xiangyu.wang@zilliz.com>

Signed-off-by: jeffoverflow zhifeng.zhang@zilliz.com

just remove standalone and milvus from $GOPATH/bin,
not every thing, on command of make clean
Signed-off-by: Nyefu.chen <yefu.chen@zilliz.com>

Signed-off-by: Nquicksilver <zhifeng.zhang@zilliz.com>

Resend unsuccessful ddMsg when master start
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* Deny jenkins slave pod to remain active for a while
Signed-off-by: Nquicksilver <zhifeng.zhang@zilliz.com>

Signed-off-by: Nquicksilver <zhifeng.zhang@zilliz.com>

Add session util doc.
Signed-off-by: Ngodchen <qingxiang.chen@zilliz.com>

- FlushMsg proto message is no longer in use

- We don't depend on "github.com/prometheus/common/log"

Signed-off-by: yangxuan xuan.yang@zilliz.com

* add datanode design doc
Signed-off-by: Nyangxuan <xuan.yang@zilliz.com>

design doc about master service recover on power failure
Signed-off-by: Nyefu.chen <yefu.chen@zilliz.com>

GetEntityByID needs the flushed segment to be sorted by RowID field, 
then do binary search to get the target id and entities.

See also: #5177 

Signed-off-by: fishpenguin kun.yu@zilliz.com

Add service registration.
Part of Issue #5174.
Signed-off-by: Ngodchen <qingxiang.chen@zilliz.com>

Signed-off-by: Nwangting0128 <ting.wang@zilliz.com>