Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
71b2d664
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
71b2d664
编写于
4月 09, 2015
作者:
M
mbalassi
提交者:
Gyula Fora
4月 10, 2015
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[streaming] ITCase for WindowWordCount
上级
36fcdae5
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
65 addition
and
6 deletion
+65
-6
flink-staging/flink-streaming/flink-streaming-examples/src/main/java/org/apache/flink/streaming/examples/windowing/WindowWordCount.java
...e/flink/streaming/examples/windowing/WindowWordCount.java
+15
-5
flink-staging/flink-streaming/flink-streaming-examples/src/test/java/org/apache/flink/streaming/examples/test/join/WindowJoinITCase.java
.../flink/streaming/examples/test/join/WindowJoinITCase.java
+0
-1
flink-staging/flink-streaming/flink-streaming-examples/src/test/java/org/apache/flink/streaming/examples/test/windowing/WindowWordCountITCase.java
...eaming/examples/test/windowing/WindowWordCountITCase.java
+50
-0
未找到文件。
flink-staging/flink-streaming/flink-streaming-examples/src/main/java/org/apache/flink/streaming/examples/windowing/WindowWordCount.java
浏览文件 @
71b2d664
...
...
@@ -46,6 +46,10 @@ import org.apache.flink.streaming.examples.wordcount.WordCount;
*/
public
class
WindowWordCount
{
// window parameters with default values
private
static
int
windowSize
=
250
;
private
static
int
slideSize
=
150
;
// *************************************************************************
// PROGRAM
// *************************************************************************
...
...
@@ -65,8 +69,8 @@ public class WindowWordCount {
DataStream
<
Tuple2
<
String
,
Integer
>>
counts
=
// split up the lines in pairs (2-tuples) containing: (word,1)
text
.
flatMap
(
new
WordCount
.
Tokenizer
())
// create windows of
250 records slided every 150
records
.
window
(
Count
.
of
(
250
)).
every
(
Count
.
of
(
150
))
// create windows of
windowSize records slided every slideSize
records
.
window
(
Count
.
of
(
windowSize
)).
every
(
Count
.
of
(
slideSize
))
// group by the tuple field "0" and sum up tuple field "1"
.
groupBy
(
0
).
sum
(
1
)
// flatten the windows to a single stream
...
...
@@ -97,17 +101,23 @@ public class WindowWordCount {
if
(
args
.
length
>
0
)
{
// parse input arguments
fileOutput
=
true
;
if
(
args
.
length
==
2
)
{
if
(
args
.
length
>=
2
&&
args
.
length
<=
4
)
{
textPath
=
args
[
0
];
outputPath
=
args
[
1
];
if
(
args
.
length
>=
3
){
windowSize
=
Integer
.
parseInt
(
args
[
2
]);
// if no slide size is specified use the
slideSize
=
args
.
length
==
3
?
windowSize
:
Integer
.
parseInt
(
args
[
2
]);
}
}
else
{
System
.
err
.
println
(
"Usage: WindowWordCount <text path> <result path>"
);
System
.
err
.
println
(
"Usage: WindowWordCount <text path> <result path>
[<window size>] [<slide size>]
"
);
return
false
;
}
}
else
{
System
.
out
.
println
(
"Executing WindowWordCount example with built-in default data."
);
System
.
out
.
println
(
" Provide parameters to read input data from a file."
);
System
.
out
.
println
(
" Usage: WindowWordCount <text path> <result path>"
);
System
.
out
.
println
(
" Usage: WindowWordCount <text path> <result path>
[<window size>] [<slide size>]
"
);
}
return
true
;
}
...
...
flink-staging/flink-streaming/flink-streaming-examples/src/test/java/org/apache/flink/streaming/examples/test/join/WindowJoinITCase.java
浏览文件 @
71b2d664
...
...
@@ -30,7 +30,6 @@ public class WindowJoinITCase extends StreamingProgramTestBase {
@Override
protected
void
preSubmit
()
throws
Exception
{
setParallelism
(
1
);
gradesPath
=
createTempFile
(
"gradesText.txt"
,
WindowJoinData
.
GRADES_INPUT
);
salariesPath
=
createTempFile
(
"salariesText.txt"
,
WindowJoinData
.
SALARIES_INPUT
);
resultPath
=
getTempDirPath
(
"result"
);
...
...
flink-staging/flink-streaming/flink-streaming-examples/src/test/java/org/apache/flink/streaming/examples/test/windowing/WindowWordCountITCase.java
0 → 100644
浏览文件 @
71b2d664
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org.apache.flink.streaming.examples.test.windowing
;
import
org.apache.flink.streaming.examples.windowing.WindowWordCount
;
import
org.apache.flink.streaming.util.StreamingProgramTestBase
;
import
org.apache.flink.test.testdata.WordCountData
;
public
class
WindowWordCountITCase
extends
StreamingProgramTestBase
{
protected
String
textPath
;
protected
String
resultPath
;
protected
String
windowSize
=
"250"
;
protected
String
slideSize
=
"150"
;
@Override
protected
void
preSubmit
()
throws
Exception
{
textPath
=
createTempFile
(
"text.txt"
,
WordCountData
.
TEXT
);
resultPath
=
getTempDirPath
(
"result"
);
}
@Override
protected
void
postSubmit
()
throws
Exception
{
// since the parallel tokenizers might have different speed
// the exact output can not be checked just whether it is well-formed
// checks that the result lines look like e.g. (faust, 2)
checkLinesAgainstRegexp
(
resultPath
,
"^\\([a-z]+,(\\d)+\\)"
);
}
@Override
protected
void
testProgram
()
throws
Exception
{
WindowWordCount
.
main
(
new
String
[]{
textPath
,
resultPath
,
windowSize
,
slideSize
});
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录