Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
BaiXuePrincess
milvus
提交
29d79134
milvus
项目概览
BaiXuePrincess
/
milvus
与 Fork 源项目一致
从无法访问的项目Fork
通知
7
Star
4
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
milvus
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
29d79134
编写于
8月 07, 2019
作者:
S
starlord
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
MS-324 Show error when there is not enough gpu memory to build index
Former-commit-id: f0dff85efafe0a74629bfc766c05387e37e8a743
上级
806dc602
变更
3
显示空白变更内容
内联
并排
Showing
3 changed file
with
42 addition
and
10 deletion
+42
-10
cpp/CHANGELOG.md
cpp/CHANGELOG.md
+4
-3
cpp/src/db/DBImpl.cpp
cpp/src/db/DBImpl.cpp
+37
-6
cpp/src/db/DBMetaImpl.cpp
cpp/src/db/DBMetaImpl.cpp
+1
-1
未找到文件。
cpp/CHANGELOG.md
浏览文件 @
29d79134
...
...
@@ -39,10 +39,11 @@ Please mark all change in change log and use the ticket from JIRA.
-
MS-261 - Update faiss version to 1.5.3 and add BUILD_FAISS_WITH_MKL as an option
-
MS-266 - Improve topk reduce time by using multi-threads
-
MS-275 - Avoid sqlite logic error excetion
-
MS-278 -
a
dd IndexStatsHelper
-
MS-278 -
A
dd IndexStatsHelper
-
MS-312 - Set openmp thread number by config
-
MS-305 - add CPU core percent metric
-
MS-310 - add milvus CPU utilization ratio and CPU/GPU temperature metrics
-
MS-305 - Add CPU core percent metric
-
MS-310 - Add milvus CPU utilization ratio and CPU/GPU temperature metrics
-
MS-324 - Show error when there is not enough gpu memory to build index
## New Feature
-
MS-180 - Add new mem manager
...
...
cpp/src/db/DBImpl.cpp
浏览文件 @
29d79134
...
...
@@ -535,11 +535,27 @@ Status DBImpl::BuildIndex(const meta::TableFileSchema& file) {
}
//step 3: build index
std
::
shared_ptr
<
ExecutionEngine
>
index
;
try
{
auto
start_time
=
METRICS_NOW_TIME
;
auto
index
=
to_index
->
BuildIndex
(
table_file
.
location_
);
index
=
to_index
->
BuildIndex
(
table_file
.
location_
);
auto
end_time
=
METRICS_NOW_TIME
;
auto
total_time
=
METRICS_MICROSECONDS
(
start_time
,
end_time
);
server
::
Metrics
::
GetInstance
().
BuildIndexDurationSecondsHistogramObserve
(
total_time
);
}
catch
(
std
::
exception
&
ex
)
{
//typical error: out of gpu memory
std
::
string
msg
=
"BuildIndex encounter exception"
+
std
::
string
(
ex
.
what
());
ENGINE_LOG_ERROR
<<
msg
;
table_file
.
file_type_
=
meta
::
TableFileSchema
::
TO_DELETE
;
status
=
meta_ptr_
->
UpdateTableFile
(
table_file
);
ENGINE_LOG_DEBUG
<<
"Failed to update file to index, mark file: "
<<
table_file
.
file_id_
<<
" to to_delete"
;
std
::
cout
<<
"ERROR: failed to build index, index file is too large or gpu memory is not enough"
<<
std
::
endl
;
return
Status
::
Error
(
msg
);
}
//step 4: if table has been deleted, dont save index file
bool
has_table
=
false
;
...
...
@@ -550,7 +566,22 @@ Status DBImpl::BuildIndex(const meta::TableFileSchema& file) {
}
//step 5: save index file
try
{
index
->
Serialize
();
}
catch
(
std
::
exception
&
ex
)
{
//typical error: out of disk space or permition denied
std
::
string
msg
=
"Serialize index encounter exception"
+
std
::
string
(
ex
.
what
());
ENGINE_LOG_ERROR
<<
msg
;
table_file
.
file_type_
=
meta
::
TableFileSchema
::
TO_DELETE
;
status
=
meta_ptr_
->
UpdateTableFile
(
table_file
);
ENGINE_LOG_DEBUG
<<
"Failed to update file to index, mark file: "
<<
table_file
.
file_id_
<<
" to to_delete"
;
std
::
cout
<<
"ERROR: failed to persist index file: "
<<
table_file
.
location_
<<
", possible out of disk space"
<<
std
::
endl
;
return
Status
::
Error
(
msg
);
}
//step 6: update meta
table_file
.
file_type_
=
meta
::
TableFileSchema
::
INDEX
;
...
...
cpp/src/db/DBMetaImpl.cpp
浏览文件 @
29d79134
...
...
@@ -932,7 +932,7 @@ Status DBMetaImpl::CleanUpFilesWithTTL(uint16_t seconds) {
table_file
.
date_
=
std
::
get
<
3
>
(
file
);
utils
::
DeleteTableFilePath
(
options_
,
table_file
);
ENGINE_LOG_DEBUG
<<
"Removing file id:"
<<
table_file
.
id_
<<
" location:"
<<
table_file
.
location_
;
ENGINE_LOG_DEBUG
<<
"Removing file id:"
<<
table_file
.
file_
id_
<<
" location:"
<<
table_file
.
location_
;
ConnectorPtr
->
remove
<
TableFileSchema
>
(
table_file
.
id_
);
}
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录