Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
apache
Iotdb
提交
e0a9f0a7
I
Iotdb
项目概览
apache
/
Iotdb
12 个月 前同步成功
通知
25
Star
3344
Fork
916
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
I
Iotdb
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
e0a9f0a7
编写于
4月 14, 2020
作者:
J
Jialin Qiao
提交者:
GitHub
4月 14, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
adjust ChunkMetadata size in run time (#1045)
上级
3e300693
变更
4
隐藏空白更改
内联
并排
Showing
4 changed file
with
28 addition
and
10 deletion
+28
-10
server/src/main/java/org/apache/iotdb/db/conf/adapter/IoTDBConfigDynamicAdapter.java
...ache/iotdb/db/conf/adapter/IoTDBConfigDynamicAdapter.java
+6
-1
server/src/main/java/org/apache/iotdb/db/engine/cache/ChunkCache.java
...ain/java/org/apache/iotdb/db/engine/cache/ChunkCache.java
+1
-1
server/src/main/java/org/apache/iotdb/db/engine/cache/ChunkMetadataCache.java
.../org/apache/iotdb/db/engine/cache/ChunkMetadataCache.java
+20
-7
server/src/main/java/org/apache/iotdb/db/engine/cache/TimeSeriesMetadataCache.java
...apache/iotdb/db/engine/cache/TimeSeriesMetadataCache.java
+1
-1
未找到文件。
server/src/main/java/org/apache/iotdb/db/conf/adapter/IoTDBConfigDynamicAdapter.java
浏览文件 @
e0a9f0a7
...
...
@@ -92,7 +92,7 @@ public class IoTDBConfigDynamicAdapter implements IDynamicAdapter {
/**
* Metadata size of per chunk, the default value is 1.5 KB.
*/
private
static
final
long
CHUNK_METADATA_SIZE_IN_BYTE
=
1536L
;
private
static
long
CHUNK_METADATA_SIZE_IN_BYTE
=
1536L
;
/**
* Average queue length in memtable pool
...
...
@@ -265,6 +265,11 @@ public class IoTDBConfigDynamicAdapter implements IDynamicAdapter {
return
totalStorageGroup
;
}
public
static
void
setChunkMetadataSizeInByte
(
long
chunkMetadataSizeInByte
)
{
CHUNK_METADATA_SIZE_IN_BYTE
=
chunkMetadataSizeInByte
;
}
/**
* Only for test
*/
...
...
server/src/main/java/org/apache/iotdb/db/engine/cache/ChunkCache.java
浏览文件 @
e0a9f0a7
...
...
@@ -64,7 +64,7 @@ public class ChunkCache {
long
currentSize
=
RamUsageEstimator
.
shallowSizeOf
(
key
)
+
RamUsageEstimator
.
sizeOf
(
value
);
averageSize
=
(
averageSize
*
count
)
+
currentSize
/
(++
count
);
return
currentSize
;
}
else
if
(
count
<
10000
)
{
}
else
if
(
count
<
10000
0
)
{
count
++;
return
averageSize
;
}
else
{
...
...
server/src/main/java/org/apache/iotdb/db/engine/cache/ChunkMetadataCache.java
浏览文件 @
e0a9f0a7
...
...
@@ -27,6 +27,7 @@ import java.util.concurrent.locks.ReentrantReadWriteLock;
import
org.apache.iotdb.db.conf.IoTDBConfig
;
import
org.apache.iotdb.db.conf.IoTDBConstant
;
import
org.apache.iotdb.db.conf.IoTDBDescriptor
;
import
org.apache.iotdb.db.conf.adapter.IoTDBConfigDynamicAdapter
;
import
org.apache.iotdb.db.engine.storagegroup.TsFileResource
;
import
org.apache.iotdb.db.query.control.FileReaderManager
;
import
org.apache.iotdb.db.utils.FileLoaderUtils
;
...
...
@@ -60,19 +61,31 @@ public class ChunkMetadataCache {
private
AtomicLong
cacheHitNum
=
new
AtomicLong
();
private
AtomicLong
cacheRequestNum
=
new
AtomicLong
();
/**
* approximate estimation of chunkMetaData size
*/
private
long
chunkMetaDataSize
=
0
;
private
ChunkMetadataCache
(
long
memoryThreshold
)
{
lruCache
=
new
LRULinkedHashMap
<
String
,
List
<
ChunkMetadata
>>(
memoryThreshold
,
true
)
{
int
count
=
0
;
long
averageChunkMetadataSize
=
0
;
@Override
protected
long
calEntrySize
(
String
key
,
List
<
ChunkMetadata
>
value
)
{
if
(
chunkMetaDataSize
==
0
&&
!
value
.
isEmpty
())
{
chunkMetaDataSize
=
RamUsageEstimator
.
sizeOf
(
value
.
get
(
0
));
if
(
value
.
isEmpty
())
{
return
key
.
getBytes
().
length
+
averageChunkMetadataSize
*
value
.
size
();
}
if
(
count
<
10
)
{
long
currentSize
=
RamUsageEstimator
.
sizeOf
(
value
.
get
(
0
));
averageChunkMetadataSize
=
((
averageChunkMetadataSize
*
count
)
+
currentSize
)
/
(++
count
);
IoTDBConfigDynamicAdapter
.
setChunkMetadataSizeInByte
(
averageChunkMetadataSize
);
return
key
.
getBytes
().
length
+
currentSize
*
value
.
size
();
}
else
if
(
count
<
100000
)
{
count
++;
return
key
.
getBytes
().
length
+
averageChunkMetadataSize
*
value
.
size
();
}
else
{
averageChunkMetadataSize
=
RamUsageEstimator
.
sizeOf
(
value
.
get
(
0
));
count
=
1
;
return
key
.
getBytes
().
length
+
averageChunkMetadataSize
*
value
.
size
();
}
return
value
.
size
()
*
chunkMetaDataSize
+
key
.
length
()
*
2
;
}
};
}
...
...
server/src/main/java/org/apache/iotdb/db/engine/cache/TimeSeriesMetadataCache.java
浏览文件 @
e0a9f0a7
...
...
@@ -66,7 +66,7 @@ public class TimeSeriesMetadataCache {
long
currentSize
=
RamUsageEstimator
.
shallowSizeOf
(
key
)
+
RamUsageEstimator
.
sizeOf
(
value
);
averageSize
=
((
averageSize
*
count
)
+
currentSize
)
/
(++
count
);
return
currentSize
;
}
else
if
(
count
<
10000
)
{
}
else
if
(
count
<
10000
0
)
{
count
++;
return
averageSize
;
}
else
{
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录