Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
BaiXuePrincess
milvus
提交
db3f092f
milvus
项目概览
BaiXuePrincess
/
milvus
与 Fork 源项目一致
从无法访问的项目Fork
通知
7
Star
4
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
milvus
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
db3f092f
编写于
4月 10, 2019
作者:
J
jinhai
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Update vector engine
Former-commit-id: 4297a3af82574e0f8c4843a6b26969b17fb13387
上级
0f307f81
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
21 addition
and
22 deletion
+21
-22
pyengine/engine/controller/vector_engine.py
pyengine/engine/controller/vector_engine.py
+21
-22
未找到文件。
pyengine/engine/controller/vector_engine.py
浏览文件 @
db3f092f
...
...
@@ -68,8 +68,19 @@ class VectorEngine(object):
pass
@
staticmethod
def
AddVectorToExistFile
(
group_name
):
pass
def
BuildVectorIndex
(
group_name
,
raw_filename
,
dimension
):
# Build index
raw_vector_array
,
raw_vector_id_array
=
VectorEngine
.
GetVectorListFromRawFile
(
group_name
)
# create index
index_builder
=
build_index
.
FactoryIndex
()
index
=
index_builder
().
build
(
dimension
,
raw_vector_array
,
raw_vector_id_array
)
# TODO(jinhai): store index into Cache
index_filename
=
raw_filename
+
'_index'
serialize
.
write_index
(
file_name
=
index_filename
,
index
=
index
)
UpdateFile
(
raw_filename
,
{
'row_number'
:
ROW_LIMIT
,
'type'
:
'index'
,
'filename'
:
index_filename
})
@
staticmethod
def
AddVector
(
group_name
,
vectors
):
...
...
@@ -90,8 +101,10 @@ class VectorEngine(object):
current_raw_file
=
file
current_raw_row_number
=
file
.
row_number
print
(
raw_filename
)
elif
file
.
type
==
'raw'
:
BuildVectorIndex
(
group_name
,
file
.
filename
,
group
.
dimension
)
else
:
p
rint
(
"---- To Build Index"
)
p
ass
else
:
pass
...
...
@@ -114,35 +127,21 @@ class VectorEngine(object):
++
table_row_number
vector_str_list
.
append
(
group_name
+
'.'
+
str
(
vector_id
))
# Build index
raw_vector_array
,
raw_vector_id_array
=
VectorEngine
.
GetVectorListFromRawFile
(
group_name
)
d
=
group
.
dimension
# create index
index_builder
=
build_index
.
FactoryIndex
()
index
=
index_builder
().
build
(
d
,
raw_vector_array
,
raw_vector_id_array
)
# TODO(jinhai): store index into Cache
index_filename
=
file
.
filename
+
'_index'
serialize
.
write_index
(
file_name
=
index_filename
,
index
=
index
)
UpdateFile
(
file
.
filename
,
{
'row_number'
:
ROW_LIMIT
,
'type'
:
'index'
,
'filename'
:
index_filename
})
BuildVectorIndex
(
group_name
,
raw_filename
,
group
.
dimension
)
# create new raw file name
raw_filename
=
str
(
group
.
file_number
+
1
)
table_row_number
=
0
current_raw_file
=
None
# update file table
MetaManager
.
CreateRawFile
(
group_name
,
raw_filename
)
# Append vectors to raw file
# If no raw file
if
current_raw_file
==
None
:
# update file table
MetaManager
.
CreateRawFile
(
group_name
,
raw_filename
)
# 1. update db
# 1. update db
on file number and row number
new_group_file_number
=
group
.
file_number
+
1
new_group_row_number
=
int
(
group
.
row_number
)
+
incoming_row_number
new_group_row_number
=
int
(
group
.
row_number
)
+
incoming_row_number
-
start_row_index
MetaManager
.
UpdateGroup
(
group_name
,
{
'file_number'
:
new_group_file_number
,
'row_number'
:
new_group_row_number
})
# 2. store vector into raw files
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录