Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
milvus
milvus
提交
6f0bbf72
M
milvus
项目概览
milvus
/
milvus
11 个月 前同步成功
通知
261
Star
22476
Fork
2472
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
milvus
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
6f0bbf72
编写于
10月 27, 2021
作者:
E
edward.zeng
提交者:
GitHub
10月 27, 2021
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[skip ci] Refine index design doc (#10752)
Signed-off-by:
N
Edward Zeng
<
jie.zeng@zilliz.com
>
上级
84b79900
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
15 addition
and
15 deletion
+15
-15
docs/design_docs/index_design.md
docs/design_docs/index_design.md
+15
-15
未找到文件。
docs/design_docs/index_design.md
浏览文件 @
6f0bbf72
...
...
@@ -22,25 +22,25 @@ In addition to service discovery, Milvus also uses etcd as a reliable meta stora
persistent status information to etcd. The purpose is to restore a certain Milvus component to its original
state after power off and restart.
## 8.2 Receive requests about index from RootCoord
inate
## 8.2 Receive requests about index from RootCoord
IndexCoord
inate receives requests from RootCoordinate
to build an index, delete an index, and query the status of an index.
IndexCoord
receives requests from RootCoord
to build an index, delete an index, and query the status of an index.
In Milvus, index building is performed asynchronously. When IndexCoord
inate
receives a request to build an index from
RootCoord
inate, it will first check whether the same index has been created according to the index parameters. If yes, it would
return the IndexBuildID of the existing task. Otherwise, it would assign a globally unique IndexBuildID to the task,
record the task in the MetaTable, write the MetaTable to etcd, and then return the IndexBuildID to RootCoord
inate.
RootCoord
inate confirms the index building was generated successfully by the IndexBuildID. At this time, the index construction
is completed yet. IndexCoord
inate starts a background process to find all the index tasks that need to be
In Milvus, index building is performed asynchronously. When IndexCoord receives a request to build an index from
RootCoord
, it will first check whether the same index has been created according to the index parameters. If yes, it would
return the IndexBuildID of the existing task. Otherwise, it would assign a globally unique IndexBuildID to the task,
record the task in the MetaTable, write the MetaTable to etcd, and then return the IndexBuildID to RootCoord
.
RootCoord
confirms the index building was generated successfully by the IndexBuildID. At this time, the index construction
is completed yet. IndexCoord
starts a background process to find all the index tasks that need to be
allocated periodically, and then allocates them to IndexNode for actual execution.
When IndexCoord
inate receives a request to delete an index from RootCoordinate, IndexCoordinate
traverses the MetaTable,
When IndexCoord
receives a request to delete an index from RootCoord, IndexCoord
traverses the MetaTable,
marks the corresponding index task as deleted, and returns. It is not really deleted from the MetaTable at this time.
IndexCoord
inate
has another background process that periodically queries the index tasks that need to be deleted.
IndexCoord has another background process that periodically queries the index tasks that need to be deleted.
When the index task is marked as deleted, and the index status is complete, the corresponding index task is actually
deleted from the MetaTable.
When IndexCoord
inate
receives a query index status request from other components, first check whether the corresponding
When IndexCoord receives a query index status request from other components, first check whether the corresponding
index task is marked for deletion in the MetaTable. If marked for deletion, the return index does not exist, otherwise,
it returns the index information.
...
...
@@ -130,13 +130,13 @@ in the MetaTable. Otherwise, only the lower version index file is cleaned up.
## 8.4 IndexNode Create Index
IndexNode is the execution node of index building tasks, and all index building tasks are forwarded to IndexNode by
IndexCoord
inate
for execution. When IndexNode executes an index build request, it first reads IndexMeta information
from etcd, and checks whether the index task is marked for deletion when IndexCoord
inate
is forwarded to IndexNode.
IndexCoord for execution. When IndexNode executes an index build request, it first reads IndexMeta information
from etcd, and checks whether the index task is marked for deletion when IndexCoord is forwarded to IndexNode.
If it is marked as deleted, then there is no need to actually build the index, just mark the index task status as
completed, and then write it to etcd. When IndexCoord
inate
perceives that the status corresponding to the index is
completed, and then write it to etcd. When IndexCoord perceives that the status corresponding to the index is
complete, it deletes the index task from the MetaTable. If it is checked that the index is not marked for deletion,
then the index needs to be built. The original data must be loaded first when building the index. The original data
is stored in MinIO/S3, and the storage path is notified by RootCoord
inate
in the index build request. After loading the
is stored in MinIO/S3, and the storage path is notified by RootCoord in the index build request. After loading the
original data, the data is deserialized into data blocks, and then cgo is called to build the index. When the index is
built, the index data is serialized into data blocks, and then written into the file. The directory organization of the
index file is "indexBuildID/IndexTaskVersion/partitionID/segmentID/key", where key corresponds to the serialized key
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录