* update mishards images to 0.7.1 (fix #1799) (#1802)

* update mishards images to 0.7.1 (fix #1799)
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* [skip ci] Update CHANGELOG
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* fix lsn meta bug (#1813)

* fix lsn meta bug
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* fix mysql lsn meta bug
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* Update CHANGELOG.md (#1814)
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Fix compile (#1815)

* Update CHANGELOG.md
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Update
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Fix compile (#1816)

* Update CHANGELOG.md
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Update
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Update
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Fix compile (#1817)

* Update CHANGELOG.md
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Update
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Update
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Increase Resources
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Decrease the compiling thread
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* #1818 Duplicate data generated after restart milvus server (#1819)

* #1818 Duplicate data generated after restart milvus server
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* set compile thread number
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* [skip-ci]Remove auto flush interval from config (#1837)
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* Fix lint
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* Fix compile error
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>
Co-authored-by: NBossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: Ngroot <yhmo@zeronedata.com>

* fix bug & solve compile warning from annoylib
Signed-off-by: Ncmli <chengming.li@zilliz.com>

* fix lint error
Signed-off-by: Ncmli <chengming.li@zilliz.com>
Co-authored-by: Ncmli <chengming.li@zilliz.com>

Signed-off-by: Nfishpenguin <kun.yu@zilliz.com>

* support GPUIVF delete
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* calc bitset offset correctly
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* support GPUSQ8 delete
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* support GPUIVFPQ delete
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update unittest for GPU delete
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* delete FAISS compile option -DFAISS_USE_FLOAT16
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* code opt
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update changelog
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* add sub/super-structure in http module
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update http test case
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* add Superstructure/Substructure in http module (fix #1784)
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* Update CHANGLOG.md
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* Update
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* Update
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* HasTable with a not exist table name, return error status
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* Server is not forbidden to create new partition which tag is _default
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* add annoy source code
Signed-off-by: Nshengjun.li <shengjun.li@zilliz.com>

* add annoy knowhere
Signed-off-by: Nshengjun.li <shengjun.li@zilliz.com>

* annoy local gtest passed
Signed-off-by: Nlichengming <chengming.li@zilliz.com>

* fix lint error and update changelog
Signed-off-by: Nlichengming <chengming.li@zilliz.com>

* fix compile error
Signed-off-by: Ncmli <chengming.li@zilliz.com>

* Update connect timeout in test cases
Signed-off-by: Nzw <zw@milvus.io>

* fix some potential bugs
Signed-off-by: Ncmli <chengming.li@zilliz.com>

* retry ci
Signed-off-by: Ncmli <chengming.li@zilliz.com>

* rerun ci!
Signed-off-by: Ncmli <chengming.li@zilliz.com>

* fix errors tested by c++ sdk
Signed-off-by: Ncmli <chengming.li@zilliz.com>

* fix lint error
Signed-off-by: Ncmli <chengming.li@zilliz.com>
Co-authored-by: Nshengjun.li <shengjun.li@zilliz.com>
Co-authored-by: Nlichengming <chengming.li@zilliz.com>
Co-authored-by: Nzw <zw@milvus.io>

Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* fix
Signed-off-by: NNicky <nicky.xj.lin@gmail.com>

* update.
Signed-off-by: NNicky <nicky.xj.lin@gmail.com>

* lint pass
Signed-off-by: Nxiaojun.lin <xiaojun.lin@zilliz.com>

* UPDATE CASE: search with tag not exist
Signed-off-by: Nzw <zw@zilliz.com>
Co-authored-by: Nzw <zw@zilliz.com>

* modify changelog
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* improve search qps
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* changelog
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* fix hang bug
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* fix combine request result bug
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* add unittest for combine request
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* fix python test failure
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* fix python test failure
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* fix unittest failure
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* #1756 fix memory exhausted during searching
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* #1756 update const names
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update mishards get file ids
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update behaviour for getting file ids
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* check default partition
Signed-off-by: Nyhz <413554850@qq.com>

* remove exception when file id not found
Signed-off-by: Nyhz <413554850@qq.com>

* Optimize behavior to get file ids from metadata in mishards (fix #1754)
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* [skip ci] Update changlog
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* #1735 fix search fail with gpu out of memory using ivf_flag
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* #1735 add lock for each GpuCacheMgr instance
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* #1735 optimize cache debug info
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* #1735 set smaller cache threshold
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* #1735 update cache debug log
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* #1735 fix cache issue
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* enable IP and fix crash
Signed-off-by: NNicky <nicky.xj.lin@gmail.com>

* update.
Signed-off-by: Nxiaojun.lin <xiaojun.lin@zilliz.com>

* lint pass
Signed-off-by: Nxiaojun.lin <xiaojun.lin@zilliz.com>

Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* #1724 Remove unused unittests
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part2
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part3
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part4
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part5
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

* support HNSW deletion on nmslib
Signed-off-by: Nlichengming <chengming.li@zilliz.com>

* update changelog
Signed-off-by: Nlichengming <chengming.li@zilliz.com>

* fix lint error on test_hnsw.cpp
Signed-off-by: Nlichengming <chengming.li@zilliz.com>
Co-authored-by: Nlichengming <chengming.li@zilliz.com>

* #1689 fix SQ8H search fail on SIFT-1B dataset
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix bug
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix gpu search fail
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix db_test
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* revert segment back
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix test_gpuresource
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix test_gpuresource
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* #267 Improve search performance: reduce delay
Signed-off-by: Njinhai <haijin.chn@gmail.com>

* #267 Improve search performance: reduce delay - update CHANGELOG
Signed-off-by: Njinhai <haijin.chn@gmail.com>

* #267 Update
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* #267 Update
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* #267 Update
Signed-off-by: Njinhai <hai.jin@zilliz.com>

* #267 Update format
Signed-off-by: NJinHai-CN <hai.jin@zilliz.com>

Signed-off-by: Njinhai <hai.jin@zilliz.com>

* update mishards to 0.7.0
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update mishards to 0.7.0
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* fix search bug and example pass
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* add new api
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* add segment stat
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* fix table info issue
Signed-off-by: Nyhz <413554850@qq.com>

* fix mishards api issue
Signed-off-by: Nyhz <413554850@qq.com>

* update mishards
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update all_in_one docker images
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* fix delete_by_id param parser & remove some comments
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update yaml config file
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update all_in_one config
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* remove delete_by_range comments
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update cmd api
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* add warning when search
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update service_handler
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update shrads requiremtns
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* [skip ci] remove surplus log info
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* #1686 API search_in_files cannot work correctly when vectors is stored in certain non-default partition
Signed-off-by: Nyhmo <yihua.mo@zilliz.com>

* #1686 API search_in_files cannot work correctly when vectors is stored in certain non-default partition
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* #1678 remove CUSTOMIZATION macro
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* disable faiss_benchmark
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* add new knowhere
Signed-off-by: NNicky <nicky.xj.lin@gmail.com>

* build pass
Signed-off-by: Nxiaojun.lin <xiaojun.lin@zilliz.com>

* update
Signed-off-by: NNicky <nicky.xj.lin@gmail.com>

* update
Signed-off-by: NNicky <nicky.xj.lin@gmail.com>

* rename algo
Signed-off-by: NNicky <nicky.xj.lin@gmail.com>

* update...
Signed-off-by: Nxiaojun.lin <xiaojun.lin@zilliz.com>

* add archive
Signed-off-by: NNicky <nicky.xj.lin@gmail.com>

* add new APIs: GetVectorById/SearchVectorById/SetBlacklist/GetBlacklist

* update unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* all unittest pass
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* test_binary pass
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* move knowhere into namespace milvus
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update hnsw
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update ConfAdapterMgr
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update ExecutionEngineImpl
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* move SetBlacklist/GetBlacklist to VecIndex
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update VectorAdapter and rename SearchById to QueryById
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update interface in ExecutionEngineImpl
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* milvus build pass
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* milvus IDMAP sdk_simple pass
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix test_server
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix test_schedule
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix CPU version Milvus build issue
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update BinaryIVF BuildAll
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update VecIndexFactory
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update ConfAdapter
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix clang-format
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix clang-format
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* update changelog
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix knowhere unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix SPTAG unittest

* fix clang-format
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix CPU version unittest build issue
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix db_test
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix test_engine
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix test_delete
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix CPU version build issue
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* change BinarySet key back
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* change IndexType to string, and add compatible API for 0.7.0
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix IndexHNSW build warning
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix clang-format
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix test_cache
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix write_index error
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* code clean
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* fix unittest
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>
Co-authored-by: NNicky <nicky.xj.lin@gmail.com>
Co-authored-by: Nxiaojun.lin <xiaojun.lin@zilliz.com>

* support IVFPQ CPU delete
Signed-off-by: Nlichengming <chengming.li@zilliz.com>

* update CHANGELOG.md
Signed-off-by: Nlichengming <chengming.li@zilliz.com>
Co-authored-by: Nlichengming <chengming.li@zilliz.com>

* add gpu cache config handler
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* remove cpu/gpu cache mgr from Config class by using cache config handler (fix #1572)
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* remove 0.8.0 from config version map
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* clean config header reference
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* fix bug in web readme
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* reduce gpu config handler to gpu resources handler
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* add engine config
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* modify handler hook(fix #1572)
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* update changlog
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* initalize value in handler by config default
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* code style format
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* fix compile error in release mode
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* resolve faiss blas threshold init in DBWrapper
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* modify cache header
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* remove comments
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* order headers
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* convert gpu res config to lower case
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* CI retry
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* adjust header order in cpu cache mar file
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* improve config test case
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* code format
Signed-off-by: NYhz <yinghao.zou@zilliz.com>

* optimize sse
Signed-off-by: Nsahuang <xiaohai.xu@zilliz.com>

* optimizer BinaryDistance
Signed-off-by: Nshengjun.li <shengjun.li@zilliz.com>

* fix superstructure
Signed-off-by: Nsahuang <xiaohai.xu@zilliz.com>
Co-authored-by: Nshengjun.li <shengjun.li@zilliz.com>
Co-authored-by: NJin Hai <hai.jin@zilliz.com>

* #1663 PQ index parameter m validation
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* typo
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* fix unittest failure
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* fix unittest failure
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* Optimize index flat L2/IP for SSE
Signed-off-by: Nsahuang <xiaohai.xu@zilliz.com>

* parallel optimization
Signed-off-by: Nsahuang <xiaohai.xu@zilliz.com>

* fix threshold
Signed-off-by: Nsahuang <xiaohai.xu@zilliz.com>

* add changelog
Signed-off-by: Nsahuang <xiaohai.xu@zilliz.com>

* add changelog
Signed-off-by: Nsahuang <xiaohai.xu@zilliz.com>
Co-authored-by: Nsahuang <xiaohai.xu@zilliz.com>

* update
Signed-off-by: NZhiru Zhu <zzhu@fandm.edu>

* update
Signed-off-by: NZhiru Zhu <zzhu@fandm.edu>

* update
Signed-off-by: NZhiru Zhu <zzhu@fandm.edu>

* #1648 The cache cannot be used all when the type is binary
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* #1646 The cache cannot be used all when the type is binary
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* #1646 The cache cannot be used all when the type is binary
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* #1651 Check validity of dimension when collection metric type is binary one
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* typo
Signed-off-by: Nyhmo <yihua.mo@zilliz.com>

* #1648 The cache cannot be used all when the type is binary
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* #1646 The cache cannot be used all when the type is binary
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* #1646 The cache cannot be used all when the type is binary
Signed-off-by: Ngroot <yihua.mo@zilliz.com>

* #1649 fix Milvus crash on old cpu
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* #1649 update debug log
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* retry CI
Signed-off-by: Nyudong.cai <yudong.cai@zilliz.com>

* add substructure & superstructure
Signed-off-by: Nshengjun.li <shengjun.li@zilliz.com>

* support c++sdk by lin.xiaojun
Signed-off-by: Nshengjun.li <shengjun.li@zilliz.com>

* optimizer omp for BinaryFlat; add tutorial
Signed-off-by: Nshengjun.li <shengjun.li@zilliz.com>

Signed-off-by: Nshengjun.li <shengjun.li@zilliz.com>
Co-authored-by: NJin Hai <hai.jin@zilliz.com>

Signed-off-by: NYhz <yinghao.zou@zilliz.com>