Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
Forever310
druid
提交
f55e1204
D
druid
项目概览
Forever310
/
druid
与 Fork 源项目一致
从无法访问的项目Fork
通知
3
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
D
druid
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
f55e1204
编写于
7月 29, 2013
作者:
F
fjy
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
fix according to code review
上级
12261902
变更
8
隐藏空白更改
内联
并排
Showing
8 changed file
with
47 addition
and
63 deletion
+47
-63
indexing-common/src/main/java/com/metamx/druid/index/QueryableIndex.java
.../src/main/java/com/metamx/druid/index/QueryableIndex.java
+3
-1
indexing-common/src/main/java/com/metamx/druid/index/SimpleQueryableIndex.java
...ain/java/com/metamx/druid/index/SimpleQueryableIndex.java
+12
-1
indexing-common/src/main/java/com/metamx/druid/index/v1/IndexIO.java
...mmon/src/main/java/com/metamx/druid/index/v1/IndexIO.java
+5
-48
indexing-common/src/main/java/com/metamx/druid/index/v1/MMappedIndex.java
...src/main/java/com/metamx/druid/index/v1/MMappedIndex.java
+20
-3
server/src/main/java/com/metamx/druid/coordination/ServerManager.java
...ain/java/com/metamx/druid/coordination/ServerManager.java
+7
-0
server/src/main/java/com/metamx/druid/loading/MMappedQueryableIndexFactory.java
...om/metamx/druid/loading/MMappedQueryableIndexFactory.java
+0
-6
server/src/main/java/com/metamx/druid/loading/QueryableIndexFactory.java
.../java/com/metamx/druid/loading/QueryableIndexFactory.java
+0
-2
server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
...in/java/com/metamx/druid/loading/SingleSegmentLoader.java
+0
-2
未找到文件。
indexing-common/src/main/java/com/metamx/druid/index/QueryableIndex.java
浏览文件 @
f55e1204
...
...
@@ -23,9 +23,11 @@ import com.metamx.druid.index.column.ColumnSelector;
import
com.metamx.druid.kv.Indexed
;
import
org.joda.time.Interval
;
import
java.io.Closeable
;
/**
*/
public
interface
QueryableIndex
extends
ColumnSelector
public
interface
QueryableIndex
extends
ColumnSelector
,
Closeable
{
public
Interval
getDataInterval
();
public
int
getNumRows
();
...
...
indexing-common/src/main/java/com/metamx/druid/index/SimpleQueryableIndex.java
浏览文件 @
f55e1204
...
...
@@ -19,10 +19,12 @@
package
com.metamx.druid.index
;
import
com.metamx.common.io.smoosh.SmooshedFileMapper
;
import
com.metamx.druid.index.column.Column
;
import
com.metamx.druid.kv.Indexed
;
import
org.joda.time.Interval
;
import
java.io.IOException
;
import
java.util.Map
;
/**
...
...
@@ -34,13 +36,15 @@ public class SimpleQueryableIndex implements QueryableIndex
private
final
Indexed
<
String
>
availableDimensions
;
private
final
Column
timeColumn
;
private
final
Map
<
String
,
Column
>
otherColumns
;
private
final
SmooshedFileMapper
fileMapper
;
public
SimpleQueryableIndex
(
Interval
dataInterval
,
Indexed
<
String
>
columnNames
,
Indexed
<
String
>
dimNames
,
Column
timeColumn
,
Map
<
String
,
Column
>
otherColumns
Map
<
String
,
Column
>
otherColumns
,
SmooshedFileMapper
fileMapper
)
{
this
.
dataInterval
=
dataInterval
;
...
...
@@ -48,6 +52,7 @@ public class SimpleQueryableIndex implements QueryableIndex
this
.
availableDimensions
=
dimNames
;
this
.
timeColumn
=
timeColumn
;
this
.
otherColumns
=
otherColumns
;
this
.
fileMapper
=
fileMapper
;
}
@Override
...
...
@@ -85,4 +90,10 @@ public class SimpleQueryableIndex implements QueryableIndex
{
return
otherColumns
.
get
(
columnName
);
}
@Override
public
void
close
()
throws
IOException
{
fileMapper
.
close
();
}
}
indexing-common/src/main/java/com/metamx/druid/index/v1/IndexIO.java
浏览文件 @
f55e1204
...
...
@@ -134,13 +134,6 @@ public class IndexIO
return
handler
.
mapDir
(
inDir
);
}
@Deprecated
public
static
void
unmapDir
(
File
inDir
)
throws
IOException
{
init
();
handler
.
close
(
inDir
);
}
public
static
QueryableIndex
loadIndex
(
File
inDir
)
throws
IOException
{
init
();
...
...
@@ -155,20 +148,6 @@ public class IndexIO
}
}
public
static
void
close
(
File
inDir
)
throws
IOException
{
init
();
final
int
version
=
getVersionFromDir
(
inDir
);
final
IndexLoader
loader
=
indexLoaders
.
get
(
version
);
if
(
loader
!=
null
)
{
loader
.
close
(
inDir
);
}
else
{
throw
new
ISE
(
"Unknown index version[%s]"
,
version
);
}
}
public
static
void
storeLatest
(
Index
index
,
File
file
)
throws
IOException
{
handler
.
storeLatest
(
index
,
file
);
...
...
@@ -282,8 +261,6 @@ public class IndexIO
public
MMappedIndex
mapDir
(
File
inDir
)
throws
IOException
;
public
void
close
(
File
inDir
)
throws
IOException
;
/**
* This only exists for some legacy compatibility reasons, Metamarkets is working on getting rid of it in
* future versions. Normal persisting of indexes is done via IndexMerger.
...
...
@@ -398,7 +375,8 @@ public class IndexIO
dimValueLookups
,
dimColumns
,
invertedIndexed
,
spatialIndexed
spatialIndexed
,
smooshedFiles
);
log
.
debug
(
"Mapped v8 index[%s] in %,d millis"
,
inDir
,
System
.
currentTimeMillis
()
-
startTime
);
...
...
@@ -406,14 +384,6 @@ public class IndexIO
return
retVal
;
}
@Override
public
void
close
(
File
inDir
)
throws
IOException
{
if
(
canBeMapped
(
inDir
))
{
Smoosh
.
close
(
inDir
);
}
}
@Override
public
void
storeLatest
(
Index
index
,
File
file
)
{
...
...
@@ -711,8 +681,6 @@ public class IndexIO
static
interface
IndexLoader
{
public
QueryableIndex
load
(
File
inDir
)
throws
IOException
;
public
void
close
(
File
inDir
)
throws
IOException
;
}
static
class
LegacyIndexLoader
implements
IndexLoader
...
...
@@ -794,15 +762,10 @@ public class IndexIO
.
setType
(
ValueType
.
LONG
)
.
setGenericColumn
(
new
LongGenericColumnSupplier
(
index
.
timestamps
))
.
build
(),
columns
columns
,
index
.
getFileMapper
()
);
}
@Override
public
void
close
(
File
inDir
)
throws
IOException
{
IndexIO
.
unmapDir
(
inDir
);
}
}
static
class
V9IndexLoader
implements
IndexLoader
...
...
@@ -834,7 +797,7 @@ public class IndexIO
}
final
QueryableIndex
index
=
new
SimpleQueryableIndex
(
dataInterval
,
cols
,
dims
,
deserializeColumn
(
mapper
,
smooshedFiles
.
mapFile
(
"__time"
)),
columns
dataInterval
,
cols
,
dims
,
deserializeColumn
(
mapper
,
smooshedFiles
.
mapFile
(
"__time"
)),
columns
,
smooshedFiles
);
log
.
debug
(
"Mapped v9 index[%s] in %,d millis"
,
inDir
,
System
.
currentTimeMillis
()
-
startTime
);
...
...
@@ -842,12 +805,6 @@ public class IndexIO
return
index
;
}
@Override
public
void
close
(
File
inDir
)
throws
IOException
{
Smoosh
.
close
(
inDir
);
}
private
Column
deserializeColumn
(
ObjectMapper
mapper
,
ByteBuffer
byteBuffer
)
throws
IOException
{
ColumnDescriptor
serde
=
mapper
.
readValue
(
...
...
indexing-common/src/main/java/com/metamx/druid/index/v1/MMappedIndex.java
浏览文件 @
f55e1204
...
...
@@ -24,18 +24,19 @@ import com.google.common.collect.Iterables;
import
com.google.common.collect.Maps
;
import
com.google.common.primitives.Ints
;
import
com.metamx.collections.spatial.ImmutableRTree
;
import
com.metamx.common.io.smoosh.SmooshedFileMapper
;
import
com.metamx.common.logger.Logger
;
import
com.metamx.druid.kv.ConciseCompressedIndexedInts
;
import
com.metamx.druid.kv.GenericIndexed
;
import
com.metamx.druid.kv.Indexed
;
import
com.metamx.druid.kv.IndexedList
;
import
com.metamx.druid.kv.IndexedLongs
;
import
com.metamx.druid.kv.IndexedRTree
;
import
com.metamx.druid.kv.VSizeIndexed
;
import
com.metamx.druid.kv.VSizeIndexedInts
;
import
it.uniroma3.mat.extendedset.intset.ImmutableConciseSet
;
import
org.joda.time.Interval
;
import
java.io.IOException
;
import
java.nio.ByteOrder
;
import
java.nio.LongBuffer
;
import
java.util.Arrays
;
...
...
@@ -57,6 +58,7 @@ public class MMappedIndex
final
Map
<
String
,
VSizeIndexed
>
dimColumns
;
final
Map
<
String
,
GenericIndexed
<
ImmutableConciseSet
>>
invertedIndexes
;
final
Map
<
String
,
ImmutableRTree
>
spatialIndexes
;
final
SmooshedFileMapper
fileMapper
;
private
final
Map
<
String
,
Integer
>
metricIndexes
=
Maps
.
newHashMap
();
...
...
@@ -69,7 +71,8 @@ public class MMappedIndex
Map
<
String
,
GenericIndexed
<
String
>>
dimValueLookups
,
Map
<
String
,
VSizeIndexed
>
dimColumns
,
Map
<
String
,
GenericIndexed
<
ImmutableConciseSet
>>
invertedIndexes
,
Map
<
String
,
ImmutableRTree
>
spatialIndexes
Map
<
String
,
ImmutableRTree
>
spatialIndexes
,
SmooshedFileMapper
fileMapper
)
{
this
.
availableDimensions
=
availableDimensions
;
...
...
@@ -81,6 +84,7 @@ public class MMappedIndex
this
.
dimColumns
=
dimColumns
;
this
.
invertedIndexes
=
invertedIndexes
;
this
.
spatialIndexes
=
spatialIndexes
;
this
.
fileMapper
=
fileMapper
;
for
(
int
i
=
0
;
i
<
availableMetrics
.
size
();
i
++)
{
metricIndexes
.
put
(
availableMetrics
.
get
(
i
),
i
);
...
...
@@ -169,6 +173,18 @@ public class MMappedIndex
return
(
retVal
==
null
)
?
emptySet
:
retVal
;
}
public
SmooshedFileMapper
getFileMapper
()
{
return
fileMapper
;
}
public
void
close
()
throws
IOException
{
if
(
fileMapper
!=
null
)
{
fileMapper
.
close
();
}
}
public
static
MMappedIndex
fromIndex
(
Index
index
)
{
log
.
info
(
"Converting timestamps"
);
...
...
@@ -273,7 +289,8 @@ public class MMappedIndex
dimValueLookups
,
dimColumns
,
invertedIndexes
,
spatialIndexes
spatialIndexes
,
null
);
}
}
server/src/main/java/com/metamx/druid/coordination/ServerManager.java
浏览文件 @
f55e1204
...
...
@@ -183,6 +183,13 @@ public class ServerManager implements QuerySegmentWalker
synchronized
(
dataSourceCounts
)
{
dataSourceCounts
.
add
(
dataSource
,
-
1L
);
}
try
{
oldQueryable
.
asQueryableIndex
().
close
();
}
catch
(
Exception
e
)
{
log
.
error
(
"Unable to close queryable index %s"
,
oldQueryable
.
getIdentifier
());
}
}
else
{
log
.
info
(
"Told to delete a queryable on dataSource[%s] for interval[%s] and version [%s] that I don't have."
,
...
...
server/src/main/java/com/metamx/druid/loading/MMappedQueryableIndexFactory.java
浏览文件 @
f55e1204
...
...
@@ -69,10 +69,4 @@ public class MMappedQueryableIndexFactory implements QueryableIndexFactory
throw
new
SegmentLoadingException
(
e
,
"%s"
,
e
.
getMessage
());
}
}
@Override
public
void
close
(
File
parentDir
)
throws
IOException
{
IndexIO
.
close
(
parentDir
);
}
}
server/src/main/java/com/metamx/druid/loading/QueryableIndexFactory.java
浏览文件 @
f55e1204
...
...
@@ -29,6 +29,4 @@ import java.io.IOException;
public
interface
QueryableIndexFactory
{
public
QueryableIndex
factorize
(
File
parentDir
)
throws
SegmentLoadingException
;
public
void
close
(
File
parentDir
)
throws
IOException
;
}
server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
浏览文件 @
f55e1204
...
...
@@ -170,8 +170,6 @@ public class SingleSegmentLoader implements SegmentLoader
log
.
info
(
"Deleting directory[%s]"
,
cacheFile
);
FileUtils
.
deleteDirectory
(
cacheFile
);
loc
.
removeSegment
(
segment
);
factory
.
close
(
cacheFile
);
}
catch
(
IOException
e
)
{
throw
new
SegmentLoadingException
(
e
,
e
.
getMessage
());
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录