Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
apache
Iotdb
提交
50d710b7
I
Iotdb
项目概览
apache
/
Iotdb
12 个月 前同步成功
通知
25
Star
3344
Fork
916
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
I
Iotdb
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
50d710b7
编写于
4月 17, 2020
作者:
J
Jialin Qiao
提交者:
GitHub
4月 17, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Avoid unnecessary close file in timed merge (#1063)
* avoid unnecessary close in timed merge
上级
a0c49d56
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
27 addition
and
14 deletion
+27
-14
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
...e/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
+2
-1
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/TsFileProcessor.java
.../apache/iotdb/db/engine/storagegroup/TsFileProcessor.java
+25
-13
未找到文件。
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
浏览文件 @
50d710b7
...
...
@@ -1349,11 +1349,12 @@ public class StorageGroupProcessor {
}
logger
.
info
(
"{} will close all files for starting a merge (fullmerge = {})"
,
storageGroupName
,
fullMerge
);
syncCloseAllWorkingTsFileProcessors
();
if
(
unSequenceFileList
.
isEmpty
()
||
sequenceFileTreeSet
.
isEmpty
())
{
logger
.
info
(
"{} no files to be merged"
,
storageGroupName
);
return
;
}
syncCloseAllWorkingTsFileProcessors
();
long
budget
=
IoTDBDescriptor
.
getInstance
().
getConfig
().
getMergeMemoryBudget
();
long
timeLowerBound
=
System
.
currentTimeMillis
()
-
dataTTL
;
...
...
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/TsFileProcessor.java
浏览文件 @
50d710b7
...
...
@@ -275,7 +275,11 @@ public class TsFileProcessor {
long
fileSize
=
tsFileResource
.
getFileSize
();
long
fileSizeThreshold
=
IoTDBDescriptor
.
getInstance
().
getConfig
()
.
getTsFileSizeThreshold
();
return
fileSize
>
fileSizeThreshold
;
if
(
fileSize
>=
fileSizeThreshold
)
{
logger
.
info
(
"{} fileSize {} >= fileSizeThreshold {}"
,
tsFileResource
.
getPath
(),
fileSize
,
fileSizeThreshold
);
}
return
fileSize
>=
fileSizeThreshold
;
}
void
syncClose
()
{
...
...
@@ -317,7 +321,20 @@ public class TsFileProcessor {
logger
.
debug
(
FLUSH_QUERY_WRITE_LOCKED
,
storageGroupName
,
tsFileResource
.
getFile
().
getName
());
}
try
{
logger
.
info
(
"Async close the file: {}"
,
tsFileResource
.
getFile
().
getAbsolutePath
());
if
(
logger
.
isInfoEnabled
())
{
if
(
workMemTable
!=
null
)
{
logger
.
info
(
"{}: flush a working memtable in async close tsfile {}, memtable size: {}, tsfile size: {}"
,
storageGroupName
,
tsFileResource
.
getFile
().
getAbsolutePath
(),
workMemTable
.
memSize
(),
tsFileResource
.
getFileSize
());
}
else
{
logger
.
info
(
"{}: flush a NotifyFlushMemTable in async close tsfile {}, tsfile size: {}"
,
storageGroupName
,
tsFileResource
.
getFile
().
getAbsolutePath
(),
tsFileResource
.
getFileSize
());
}
}
if
(
shouldClose
)
{
return
;
}
...
...
@@ -332,11 +349,7 @@ public class TsFileProcessor {
//we have to add the memtable into flushingList first and then set the shouldClose tag.
// see https://issues.apache.org/jira/browse/IOTDB-510
IMemTable
tmpMemTable
=
workMemTable
==
null
?
new
NotifyFlushMemTable
()
:
workMemTable
;
if
(
logger
.
isDebugEnabled
())
{
logger
.
debug
(
"{}: {} async flush a memtable (signal = {}) when async close"
,
storageGroupName
,
tsFileResource
.
getFile
().
getName
(),
tmpMemTable
.
isSignalMemTable
());
}
try
{
addAMemtableIntoFlushingList
(
tmpMemTable
);
shouldClose
=
true
;
...
...
@@ -575,7 +588,7 @@ public class TsFileProcessor {
endFile
();
}
catch
(
Exception
e
)
{
logger
.
error
(
"{} meet error when flush FileMetadata to {}, change system mode to read-only"
,
storageGroupName
,
tsFileResource
.
getFile
().
getAbsolutePath
());
storageGroupName
,
tsFileResource
.
getFile
().
getAbsolutePath
()
,
e
);
IoTDBDescriptor
.
getInstance
().
getConfig
().
setReadOnly
(
true
);
try
{
writer
.
reset
();
...
...
@@ -607,17 +620,16 @@ public class TsFileProcessor {
// mark the TsFileResource closed, no need writer anymore
closeTsFileCallback
.
call
(
this
);
writer
=
null
;
if
(
logger
.
isInfoEnabled
())
{
long
closeEndTime
=
System
.
currentTimeMillis
();
logger
.
info
(
"Storage group {} close the file {},
start time is {}, end tim
e is {}, "
logger
.
info
(
"Storage group {} close the file {},
TsFile siz
e is {}, "
+
"time consumption of flushing metadata is {}ms"
,
storageGroupName
,
tsFileResource
.
getFile
().
getAbsoluteFile
(),
DatetimeUtils
.
convertMillsecondToZonedDateTime
(
closeStartTime
),
DatetimeUtils
.
convertMillsecondToZonedDateTime
(
closeEndTime
),
writer
.
getFile
().
length
(),
closeEndTime
-
closeStartTime
);
}
writer
=
null
;
}
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录