Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
Forever310
druid
提交
13886069
D
druid
项目概览
Forever310
/
druid
与 Fork 源项目一致
从无法访问的项目Fork
通知
3
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
D
druid
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
13886069
编写于
8月 21, 2013
作者:
C
cheddar
浏览文件
操作
浏览文件
下载
差异文件
Merge pull request #225 from metamx/num-shard-spec
NumberedShardSpec
上级
b4310d6c
c56c6b8b
变更
5
隐藏空白更改
内联
并排
Showing
5 changed file
with
307 addition
and
3 deletion
+307
-3
client/src/main/java/com/metamx/druid/shard/NumberedShardSpec.java
...c/main/java/com/metamx/druid/shard/NumberedShardSpec.java
+82
-0
client/src/main/java/com/metamx/druid/shard/ShardSpec.java
client/src/main/java/com/metamx/druid/shard/ShardSpec.java
+2
-1
client/src/test/java/com/metamx/druid/shard/NumberedShardSpecTest.java
...st/java/com/metamx/druid/shard/NumberedShardSpecTest.java
+103
-0
common/src/main/java/com/metamx/druid/partition/IntegerPartitionChunk.java
...ava/com/metamx/druid/partition/IntegerPartitionChunk.java
+3
-2
common/src/main/java/com/metamx/druid/partition/NumberedPartitionChunk.java
...va/com/metamx/druid/partition/NumberedPartitionChunk.java
+117
-0
未找到文件。
client/src/main/java/com/metamx/druid/shard/NumberedShardSpec.java
0 → 100644
浏览文件 @
13886069
/*
* Druid - a distributed column store.
* Copyright (C) 2012 Metamarkets Group Inc.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*/
package
com.metamx.druid.shard
;
import
com.fasterxml.jackson.annotation.JsonCreator
;
import
com.fasterxml.jackson.annotation.JsonIgnore
;
import
com.fasterxml.jackson.annotation.JsonProperty
;
import
com.google.common.base.Preconditions
;
import
com.metamx.druid.input.InputRow
;
import
com.metamx.druid.partition.NumberedPartitionChunk
;
import
com.metamx.druid.partition.PartitionChunk
;
import
java.util.Map
;
public
class
NumberedShardSpec
implements
ShardSpec
{
@JsonIgnore
final
private
int
partitionNum
;
@JsonIgnore
final
private
int
partitions
;
@JsonCreator
public
NumberedShardSpec
(
@JsonProperty
(
"partitionNum"
)
int
partitionNum
,
@JsonProperty
(
"partitions"
)
int
partitions
)
{
Preconditions
.
checkArgument
(
partitionNum
>=
0
,
"partitionNum >= 0"
);
Preconditions
.
checkArgument
(
partitionNum
<
partitions
,
"partitionNum < partitions"
);
this
.
partitionNum
=
partitionNum
;
this
.
partitions
=
partitions
;
}
@JsonProperty
(
"partitionNum"
)
@Override
public
int
getPartitionNum
()
{
return
partitionNum
;
}
@JsonProperty
(
"partitions"
)
public
int
getPartitions
()
{
return
partitions
;
}
@Override
public
<
T
>
PartitionChunk
<
T
>
createChunk
(
T
obj
)
{
return
NumberedPartitionChunk
.
make
(
partitionNum
,
partitions
,
obj
);
}
@Override
public
boolean
isInChunk
(
Map
<
String
,
String
>
dimensions
)
{
return
true
;
}
@Override
public
boolean
isInChunk
(
InputRow
inputRow
)
{
return
true
;
}
}
client/src/main/java/com/metamx/druid/shard/ShardSpec.java
浏览文件 @
13886069
...
@@ -33,7 +33,8 @@ import java.util.Map;
...
@@ -33,7 +33,8 @@ import java.util.Map;
@JsonSubTypes
({
@JsonSubTypes
({
@JsonSubTypes
.
Type
(
name
=
"single"
,
value
=
SingleDimensionShardSpec
.
class
),
@JsonSubTypes
.
Type
(
name
=
"single"
,
value
=
SingleDimensionShardSpec
.
class
),
@JsonSubTypes
.
Type
(
name
=
"none"
,
value
=
NoneShardSpec
.
class
),
@JsonSubTypes
.
Type
(
name
=
"none"
,
value
=
NoneShardSpec
.
class
),
@JsonSubTypes
.
Type
(
name
=
"linear"
,
value
=
LinearShardSpec
.
class
)
@JsonSubTypes
.
Type
(
name
=
"linear"
,
value
=
LinearShardSpec
.
class
),
@JsonSubTypes
.
Type
(
name
=
"numbered"
,
value
=
NumberedShardSpec
.
class
)
})
})
public
interface
ShardSpec
public
interface
ShardSpec
{
{
...
...
client/src/test/java/com/metamx/druid/shard/NumberedShardSpecTest.java
0 → 100644
浏览文件 @
13886069
/*
* Druid - a distributed column store.
* Copyright (C) 2012 Metamarkets Group Inc.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*/
package
com.metamx.druid.shard
;
import
com.fasterxml.jackson.databind.ObjectMapper
;
import
com.google.common.base.Function
;
import
com.google.common.collect.ImmutableList
;
import
com.google.common.collect.Lists
;
import
com.metamx.druid.jackson.DefaultObjectMapper
;
import
com.metamx.druid.partition.PartitionChunk
;
import
junit.framework.Assert
;
import
org.junit.Test
;
import
java.util.List
;
public
class
NumberedShardSpecTest
{
@Test
public
void
testSerdeRoundTrip
()
throws
Exception
{
final
ObjectMapper
jsonMapper
=
new
DefaultObjectMapper
();
final
ShardSpec
spec
=
jsonMapper
.
readValue
(
jsonMapper
.
writeValueAsBytes
(
new
NumberedShardSpec
(
1
,
2
)),
ShardSpec
.
class
);
Assert
.
assertEquals
(
1
,
spec
.
getPartitionNum
());
Assert
.
assertEquals
(
2
,
((
NumberedShardSpec
)
spec
).
getPartitions
());
}
@Test
public
void
testSerdeBackwardsCompat
()
throws
Exception
{
final
ObjectMapper
jsonMapper
=
new
DefaultObjectMapper
();
final
ShardSpec
spec
=
jsonMapper
.
readValue
(
"{\"type\": \"numbered\", \"partitions\": 2, \"partitionNum\": 1}"
,
ShardSpec
.
class
);
Assert
.
assertEquals
(
1
,
spec
.
getPartitionNum
());
Assert
.
assertEquals
(
2
,
((
NumberedShardSpec
)
spec
).
getPartitions
());
}
@Test
public
void
testPartitionChunks
()
{
final
List
<
ShardSpec
>
specs
=
ImmutableList
.<
ShardSpec
>
of
(
new
NumberedShardSpec
(
0
,
3
),
new
NumberedShardSpec
(
1
,
3
),
new
NumberedShardSpec
(
2
,
3
)
);
final
List
<
PartitionChunk
<
String
>>
chunks
=
Lists
.
transform
(
specs
,
new
Function
<
ShardSpec
,
PartitionChunk
<
String
>>()
{
@Override
public
PartitionChunk
<
String
>
apply
(
ShardSpec
shardSpec
)
{
return
shardSpec
.
createChunk
(
"rofl"
);
}
}
);
Assert
.
assertEquals
(
0
,
chunks
.
get
(
0
).
getChunkNumber
());
Assert
.
assertEquals
(
1
,
chunks
.
get
(
1
).
getChunkNumber
());
Assert
.
assertEquals
(
2
,
chunks
.
get
(
2
).
getChunkNumber
());
Assert
.
assertTrue
(
chunks
.
get
(
0
).
isStart
());
Assert
.
assertFalse
(
chunks
.
get
(
1
).
isStart
());
Assert
.
assertFalse
(
chunks
.
get
(
2
).
isStart
());
Assert
.
assertFalse
(
chunks
.
get
(
0
).
isEnd
());
Assert
.
assertFalse
(
chunks
.
get
(
1
).
isEnd
());
Assert
.
assertTrue
(
chunks
.
get
(
2
).
isEnd
());
Assert
.
assertTrue
(
chunks
.
get
(
0
).
abuts
(
chunks
.
get
(
1
)));
Assert
.
assertTrue
(
chunks
.
get
(
1
).
abuts
(
chunks
.
get
(
2
)));
Assert
.
assertFalse
(
chunks
.
get
(
0
).
abuts
(
chunks
.
get
(
0
)));
Assert
.
assertFalse
(
chunks
.
get
(
0
).
abuts
(
chunks
.
get
(
2
)));
Assert
.
assertFalse
(
chunks
.
get
(
1
).
abuts
(
chunks
.
get
(
0
)));
Assert
.
assertFalse
(
chunks
.
get
(
1
).
abuts
(
chunks
.
get
(
1
)));
Assert
.
assertFalse
(
chunks
.
get
(
2
).
abuts
(
chunks
.
get
(
0
)));
Assert
.
assertFalse
(
chunks
.
get
(
2
).
abuts
(
chunks
.
get
(
1
)));
Assert
.
assertFalse
(
chunks
.
get
(
2
).
abuts
(
chunks
.
get
(
2
)));
}
}
common/src/main/java/com/metamx/druid/partition/IntegerPartitionChunk.java
浏览文件 @
13886069
...
@@ -96,10 +96,10 @@ public class IntegerPartitionChunk<T> implements PartitionChunk<T>
...
@@ -96,10 +96,10 @@ public class IntegerPartitionChunk<T> implements PartitionChunk<T>
{
{
if
(
chunk
instanceof
IntegerPartitionChunk
)
{
if
(
chunk
instanceof
IntegerPartitionChunk
)
{
IntegerPartitionChunk
<
T
>
intChunk
=
(
IntegerPartitionChunk
<
T
>)
chunk
;
IntegerPartitionChunk
<
T
>
intChunk
=
(
IntegerPartitionChunk
<
T
>)
chunk
;
return
comparator
.
compare
(
chunkNumber
,
intChunk
.
chunkNumber
);
return
comparator
.
compare
(
chunkNumber
,
intChunk
.
chunkNumber
);
}
else
{
throw
new
IllegalArgumentException
(
"Cannot compare against something that is not an IntegerPartitionChunk."
);
}
}
throw
new
IllegalArgumentException
(
"Cannot compare against something that is not a StringPartitionChunk."
);
}
}
@Override
@Override
...
@@ -134,3 +134,4 @@ public class IntegerPartitionChunk<T> implements PartitionChunk<T>
...
@@ -134,3 +134,4 @@ public class IntegerPartitionChunk<T> implements PartitionChunk<T>
'}'
;
'}'
;
}
}
}
}
common/src/main/java/com/metamx/druid/partition/NumberedPartitionChunk.java
0 → 100644
浏览文件 @
13886069
/*
* Druid - a distributed column store.
* Copyright (C) 2012 Metamarkets Group Inc.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*/
package
com.metamx.druid.partition
;
import
com.google.common.base.Objects
;
import
com.google.common.base.Preconditions
;
import
com.google.common.collect.ComparisonChain
;
public
class
NumberedPartitionChunk
<
T
>
implements
PartitionChunk
<
T
>
{
private
final
int
chunkNumber
;
private
final
int
chunks
;
private
final
T
object
;
public
static
<
T
>
NumberedPartitionChunk
<
T
>
make
(
int
chunkNumber
,
int
chunks
,
T
obj
)
{
return
new
NumberedPartitionChunk
<
T
>(
chunkNumber
,
chunks
,
obj
);
}
public
NumberedPartitionChunk
(
int
chunkNumber
,
int
chunks
,
T
object
)
{
Preconditions
.
checkArgument
(
chunkNumber
>=
0
,
"chunkNumber >= 0"
);
Preconditions
.
checkArgument
(
chunkNumber
<
chunks
,
"chunkNumber < chunks"
);
this
.
chunkNumber
=
chunkNumber
;
this
.
chunks
=
chunks
;
this
.
object
=
object
;
}
@Override
public
T
getObject
()
{
return
object
;
}
@Override
public
boolean
abuts
(
final
PartitionChunk
<
T
>
other
)
{
return
other
instanceof
NumberedPartitionChunk
&&
other
.
getChunkNumber
()
==
chunkNumber
+
1
;
}
@Override
public
boolean
isStart
()
{
return
chunkNumber
==
0
;
}
@Override
public
boolean
isEnd
()
{
return
chunkNumber
==
chunks
-
1
;
}
@Override
public
int
getChunkNumber
()
{
return
chunkNumber
;
}
@Override
public
int
compareTo
(
PartitionChunk
<
T
>
other
)
{
if
(
other
instanceof
NumberedPartitionChunk
)
{
final
NumberedPartitionChunk
castedOther
=
(
NumberedPartitionChunk
)
other
;
return
ComparisonChain
.
start
()
.
compare
(
chunks
,
castedOther
.
chunks
)
.
compare
(
chunkNumber
,
castedOther
.
chunkNumber
)
.
result
();
}
else
{
throw
new
IllegalArgumentException
(
"Cannot compare against something that is not a NumberedPartitionChunk."
);
}
}
@Override
@SuppressWarnings
(
"unchecked"
)
public
boolean
equals
(
Object
o
)
{
if
(
this
==
o
)
{
return
true
;
}
if
(
o
==
null
||
getClass
()
!=
o
.
getClass
())
{
return
false
;
}
return
compareTo
((
NumberedPartitionChunk
<
T
>)
o
)
==
0
;
}
@Override
public
int
hashCode
()
{
return
Objects
.
hashCode
(
chunks
,
chunkNumber
);
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录