Mishards 010 upgrade & Add config unit in demo config (#2550)

* Update mishards config
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* Update mishards config
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* Upgrade mishards for milvus v0.10.0 (fix #2548)
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* Launch server fail using demo config (#2549)
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* [skip ci] Remove surplus config yaml
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

CHANGELOG.md

@@ -8,6 +8,7 @@ Please mark all change in change log and use the issue from GitHub
 -   \#2367 Fix inconsistent reading and writing when using mishards
 -   \#2368 Make read node detect delete behavior
 -   \#2394 Drop collection timeout if too many partitions created on collection
+-   \#2549 Launch server fail using demo config
 
 ## Feature
 -   \#2363 Update branch version
@@ -21,6 +22,7 @@ Please mark all change in change log and use the issue from GitHub
 -   \#2489 Add exception throw on mysql meta error
 -   \#2495 Add creating lock file failure reason.
 -   \#2516 Improve unit test coverage
+-   \#2548 Upgrade mishards for milvus v0.10.0
 
 ## Task
 

core/conf/demo/server_config.yaml

@@ -95,7 +95,7 @@ storage:
 wal:
   enable: true
   recovery_error_ignore: false
-  buffer_size: 256
+  buffer_size: 256MB
   path: /var/lib/milvus/wal
 
 #----------------------+------------------------------------------------------------+------------+-----------------+
@@ -115,8 +115,8 @@ wal:
 #                      | double-quote required).                                    |            |                 |
 #----------------------+------------------------------------------------------------+------------+-----------------+
 cache:
-  cache_size: 4
-  insert_buffer_size: 1
+  cache_size: 4GB
+  insert_buffer_size: 1GB
   preload_collection:
 
 #----------------------+------------------------------------------------------------+------------+-----------------+
@@ -142,7 +142,7 @@ cache:
 #----------------------+------------------------------------------------------------+------------+-----------------+
 gpu:
   enable: false
-  cache_size: 1
+  cache_size: 1GB
   gpu_search_threshold: 1000
   search_devices:
     - gpu0
@@ -168,7 +168,7 @@ logs:
   level: debug
   trace.enable: true
   path: /var/lib/milvus/logs
-  max_log_file_size: 1024
+  max_log_file_size: 1024MB
   log_rotate_num: 0
 
 #----------------------+------------------------------------------------------------+------------+-----------------+

shards/all_in_one/all_in_one.yml

@@ -3,7 +3,7 @@ services:
     milvus_wr:
         runtime: nvidia
         restart: always
-        image: milvusdb/milvus:0.9.1-gpu-d052920-e04ed5
+        image: milvusdb/milvus:0.10.0-gpu-d061320-83cc81
         ports:
             - "0.0.0.0:19540:19530"
         volumes:
@@ -13,7 +13,7 @@ services:
     milvus_ro:
         runtime: nvidia
         restart: always
-        image: milvusdb/milvus:0.9.1-gpu-d052920-e04ed5
+        image: milvusdb/milvus:0.10.0-gpu-d061320-83cc81
         ports:
             - "0.0.0.0:19541:19530"
         volumes:

shards/all_in_one_with_mysql/all_in_one.yml

@@ -18,7 +18,7 @@ services:
     milvus_wr:
         runtime: nvidia
         restart: always
-        image: milvusdb/milvus:0.9.1-gpu-d052920-e04ed5
+        image: milvusdb/milvus:0.10.0-gpu-d061320-83cc81
         volumes:
             - /tmp/milvus/db:/var/lib/milvus/db
             - ./wr_server.yml:/var/lib/milvus/conf/server_config.yaml
@@ -29,7 +29,7 @@ services:
     milvus_ro:
         runtime: nvidia
         restart: always
-        image: milvusdb/milvus:0.9.1-gpu-d052920-e04ed5
+        image: milvusdb/milvus:0.10.0-gpu-d061320-83cc81
         volumes:
             - /tmp/milvus/db:/var/lib/milvus/db
             - ./ro_server.yml:/var/lib/milvus/conf/server_config.yaml

shards/mishards/connections.py

+import copy
 import time
 import json
 import logging
@@ -5,7 +6,6 @@ import threading
 from functools import wraps
 from collections import defaultdict
 from milvus import Milvus
-# from milvus.client.hooks import BaseSearchHook
 
 from mishards import (settings, exceptions, topology)
 from utils import singleton
@@ -254,7 +254,9 @@ class ConnectionGroup(topology.TopoGroup):
         uri = kwargs.get('uri', None)
         if not uri:
             raise RuntimeError('\"uri\" is required to create connection pool')
-        pool = Milvus(name=name, **kwargs)
+        milvus_args = copy.deepcopy(kwargs)
+        milvus_args["max_retry"] = settings.MAX_RETRY
+        pool = Milvus(name=name, **milvus_args)
         status = self.add(pool)
         if status != topology.StatusType.OK:
             pool = None

shards/requirements.txt

@@ -14,8 +14,8 @@ py==1.8.0
 pyasn1==0.4.7
 pyasn1-modules==0.2.6
 pylint==2.5.0
-#pymilvus==0.2.12
-pymilvus-test==0.3.25
+pymilvus==0.2.13
+#pymilvus-test==0.3.25
 pyparsing==2.4.0
 pytest==4.6.3
 pytest-level==0.1.1