Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
233c0147
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
233c0147
编写于
2月 02, 2016
作者:
C
chengxiang li
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[FLINK-3281] IndexOutOfBoundsException when range partition on empty DataSet
上级
6c0a83e4
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
41 addition
and
7 deletion
+41
-7
flink-runtime/src/main/java/org/apache/flink/runtime/operators/udf/RangeBoundaryBuilder.java
...che/flink/runtime/operators/udf/RangeBoundaryBuilder.java
+9
-7
flink-tests/src/test/scala/org/apache/flink/api/scala/operators/PartitionITCase.scala
...rg/apache/flink/api/scala/operators/PartitionITCase.scala
+32
-0
未找到文件。
flink-runtime/src/main/java/org/apache/flink/runtime/operators/udf/RangeBoundaryBuilder.java
浏览文件 @
233c0147
...
...
@@ -60,13 +60,15 @@ public class RangeBoundaryBuilder<T> extends RichMapPartitionFunction<T, Object[
int
boundarySize
=
parallelism
-
1
;
Object
[][]
boundaries
=
new
Object
[
boundarySize
][];
double
avgRange
=
sampledData
.
size
()
/
(
double
)
parallelism
;
int
numKey
=
comparator
.
getFlatComparators
().
length
;
for
(
int
i
=
1
;
i
<
parallelism
;
i
++)
{
T
record
=
sampledData
.
get
((
int
)
(
i
*
avgRange
));
Object
[]
keys
=
new
Object
[
numKey
];
comparator
.
extractKeys
(
record
,
keys
,
0
);
boundaries
[
i
-
1
]
=
keys
;
if
(
sampledData
.
size
()
>
0
)
{
double
avgRange
=
sampledData
.
size
()
/
(
double
)
parallelism
;
int
numKey
=
comparator
.
getFlatComparators
().
length
;
for
(
int
i
=
1
;
i
<
parallelism
;
i
++)
{
T
record
=
sampledData
.
get
((
int
)
(
i
*
avgRange
));
Object
[]
keys
=
new
Object
[
numKey
];
comparator
.
extractKeys
(
record
,
keys
,
0
);
boundaries
[
i
-
1
]
=
keys
;
}
}
out
.
collect
(
boundaries
);
...
...
flink-tests/src/test/scala/org/apache/flink/api/scala/operators/PartitionITCase.scala
浏览文件 @
233c0147
...
...
@@ -49,6 +49,38 @@ class PartitionITCase(mode: TestExecutionMode) extends MultipleProgramsTestBase(
TestBaseUtils
.
compareResultsByLinesInMemory
(
expected
,
resultPath
)
}
@Test
def
testEmptyHashPartition
()
:
Unit
=
{
/*
* Test hash partition by tuple field
*/
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
ds
=
env
.
fromCollection
(
Seq
[
Tuple1
[
String
]]())
val
unique
=
ds
.
partitionByHash
(
0
)
unique
.
writeAsText
(
resultPath
,
WriteMode
.
OVERWRITE
)
env
.
execute
()
expected
=
""
}
@Test
def
testEmptyRangePartition
()
:
Unit
=
{
/*
* Test hash partition by tuple field
*/
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
ds
=
env
.
fromCollection
(
Seq
[
Tuple1
[
String
]]())
val
unique
=
ds
.
partitionByRange
(
0
)
unique
.
writeAsText
(
resultPath
,
WriteMode
.
OVERWRITE
)
env
.
execute
()
expected
=
""
}
@Test
def
testHashPartitionByTupleField
()
:
Unit
=
{
/*
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录