Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
ecb5595c
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
ecb5595c
编写于
1月 10, 2015
作者:
M
mbalassi
提交者:
Stephan Ewen
1月 11, 2015
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[FLINK-1358] [streaming] Streaming added to the quickstarts
上级
86f87537
变更
10
隐藏空白更改
内联
并排
Showing
10 changed file
with
191 addition
and
8 deletion
+191
-8
flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml
...start-java/src/main/resources/archetype-resources/pom.xml
+5
-0
flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/SocketTextStreamWordCount.java
...pe-resources/src/main/java/SocketTextStreamWordCount.java
+107
-0
flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/WordCount.java
...esources/archetype-resources/src/main/java/WordCount.java
+3
-5
flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml
...tart-scala/src/main/resources/archetype-resources/pom.xml
+5
-0
flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/SocketTextStreamWordCount.scala
...-resources/src/main/scala/SocketTextStreamWordCount.scala
+66
-0
flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/WordCount.scala
...ources/archetype-resources/src/main/scala/WordCount.scala
+1
-1
flink-quickstart/quickstart-SNAPSHOT.sh
flink-quickstart/quickstart-SNAPSHOT.sh
+1
-0
flink-quickstart/quickstart-scala-SNAPSHOT.sh
flink-quickstart/quickstart-scala-SNAPSHOT.sh
+1
-0
flink-quickstart/quickstart-scala.sh
flink-quickstart/quickstart-scala.sh
+1
-1
flink-quickstart/quickstart.sh
flink-quickstart/quickstart.sh
+1
-1
未找到文件。
flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml
浏览文件 @
ecb5595c
...
...
@@ -54,6 +54,11 @@ under the License.
<artifactId>
flink-java
</artifactId>
<version>
0.9-SNAPSHOT
</version>
</dependency>
<dependency>
<groupId>
org.apache.flink
</groupId>
<artifactId>
flink-streaming-core
</artifactId>
<version>
0.9-SNAPSHOT
</version>
</dependency>
<dependency>
<groupId>
org.apache.flink
</groupId>
<artifactId>
flink-clients
</artifactId>
...
...
flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/SocketTextStreamWordCount.java
0 → 100644
浏览文件 @
ecb5595c
package
${
package
};
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import
org.apache.flink.api.common.functions.FlatMapFunction
;
import
org.apache.flink.api.java.tuple.Tuple2
;
import
org.apache.flink.streaming.api.datastream.DataStream
;
import
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
;
import
org.apache.flink.util.Collector
;
/**
* This example shows an implementation of WordCount with data from a text
* socket. To run the example make sure that the service providing the text data
* is already up and running.
*
* <p>
* To start an example socket text stream on your local machine run netcat from
* a command line: <code>nc -lk 9999</code>, where the parameter specifies the
* port number.
*
*
* <p>
* Usage:
* <code>SocketTextStreamWordCount <hostname> <port> <result path></code>
* <br>
*
* <p>
* This example shows how to:
* <ul>
* <li>use StreamExecutionEnvironment.socketTextStream
* <li>write a simple Flink program,
* <li>write and use user-defined functions.
* </ul>
*
* @see <a href="www.openbsd.org/cgi-bin/man.cgi?query=nc">netcat</a>
*/
public
class
SocketTextStreamWordCount
{
//
// Program
//
public
static
void
main
(
String
[]
args
)
throws
Exception
{
String
hostName
=
args
[
0
];
Integer
port
=
Integer
.
parseInt
(
args
[
1
]);
// set up the execution environment
final
StreamExecutionEnvironment
env
=
StreamExecutionEnvironment
.
getExecutionEnvironment
();
// get input data
DataStream
<
String
>
text
=
env
.
socketTextStream
(
hostName
,
port
);
DataStream
<
Tuple2
<
String
,
Integer
>>
counts
=
// split up the lines in pairs (2-tuples) containing: (word,1)
text
.
flatMap
(
new
LineSplitter
())
// group by the tuple field "0" and sum up tuple field "1"
.
groupBy
(
0
)
.
sum
(
1
);
counts
.
print
();
// execute program
env
.
execute
(
"WordCount from SocketTextStream Example"
);
}
//
// User Functions
//
/**
* Implements the string tokenizer that splits sentences into words as a user-defined
* FlatMapFunction. The function takes a line (String) and splits it into
* multiple pairs in the form of "(word,1)" (Tuple2<String, Integer>).
*/
public
static
final
class
LineSplitter
implements
FlatMapFunction
<
String
,
Tuple2
<
String
,
Integer
>>
{
@Override
public
void
flatMap
(
String
value
,
Collector
<
Tuple2
<
String
,
Integer
>>
out
)
{
// normalize and split the line
String
[]
tokens
=
value
.
toLowerCase
().
split
(
"\\W+"
);
// emit the pairs
for
(
String
token
:
tokens
)
{
if
(
token
.
length
()
>
0
)
{
out
.
collect
(
new
Tuple2
<
String
,
Integer
>(
token
,
1
));
}
}
}
}
}
flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/WordCount
Job
.java
→
flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/src/main/java/WordCount.java
浏览文件 @
ecb5595c
...
...
@@ -20,7 +20,6 @@ package ${package};
import
org.apache.flink.api.java.DataSet
;
import
org.apache.flink.api.java.ExecutionEnvironment
;
import
org.apache.flink.api.java.aggregation.Aggregations
;
import
org.apache.flink.api.common.functions.FlatMapFunction
;
import
org.apache.flink.api.java.tuple.Tuple2
;
import
org.apache.flink.util.Collector
;
...
...
@@ -38,11 +37,10 @@ import org.apache.flink.util.Collector;
* </ul>
*
*/
@SuppressWarnings
(
"serial"
)
public
class
WordCountJob
{
public
class
WordCount
{
//
// Program
.
// Program
//
public
static
void
main
(
String
[]
args
)
throws
Exception
{
...
...
@@ -63,7 +61,7 @@ public class WordCountJob {
text
.
flatMap
(
new
LineSplitter
())
// group by the tuple field "0" and sum up tuple field "1"
.
groupBy
(
0
)
.
aggregate
(
Aggregations
.
SUM
,
1
);
.
sum
(
1
);
// emit result
counts
.
print
();
...
...
flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml
浏览文件 @
ecb5595c
...
...
@@ -54,6 +54,11 @@ under the License.
<artifactId>
flink-scala
</artifactId>
<version>
0.9-SNAPSHOT
</version>
</dependency>
<dependency>
<groupId>
org.apache.flink
</groupId>
<artifactId>
flink-streaming-scala
</artifactId>
<version>
0.9-SNAPSHOT
</version>
</dependency>
<dependency>
<groupId>
org.apache.flink
</groupId>
<artifactId>
flink-clients
</artifactId>
...
...
flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/SocketTextStreamWordCount.scala
0 → 100644
浏览文件 @
ecb5595c
package
$
{
package
}
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import
org.apache.flink.streaming.api.scala._
/**
* This example shows an implementation of WordCount with data from a text socket.
* To run the example make sure that the service providing the text data is already up and running.
*
* To start an example socket text stream on your local machine run netcat from a command line,
* where the parameter specifies the port number:
*
* {{{
* nc -lk 9999
* }}}
*
* Usage:
* {{{
* SocketTextStreamWordCount <hostname> <port> <output path>
* }}}
*
* This example shows how to:
*
* - use StreamExecutionEnvironment.socketTextStream
* - write a simple Flink Streaming program in scala.
* - write and use user-defined functions.
*/
object
SocketTextStreamWordCount
{
def
main
(
args
:
Array
[
String
])
{
val
hostName
=
args
(
0
)
val
port
=
args
(
1
).
toInt
val
env
=
StreamExecutionEnvironment
.
getExecutionEnvironment
//Create streams for names and ages by mapping the inputs to the corresponding objects
val
text
=
env
.
socketTextStream
(
hostName
,
port
)
val
counts
=
text
.
flatMap
{
_
.
toLowerCase
.
split
(
"\\W+"
)
filter
{
_
.
nonEmpty
}
}
.
map
{
(
_
,
1
)
}
.
groupBy
(
0
)
.
sum
(
1
)
counts
print
env
.
execute
(
"Scala SocketTextStreamWordCount Example"
)
}
}
flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/WordCount
Job
.scala
→
flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/src/main/scala/WordCount.scala
浏览文件 @
ecb5595c
...
...
@@ -30,7 +30,7 @@ import org.apache.flink.api.scala._
* - use Tuple data types.
* - write and use user-defined functions.
*/
object
WordCount
Job
{
object
WordCount
{
def
main
(
args
:
Array
[
String
])
{
// set up the execution environment
...
...
flink-quickstart/quickstart-SNAPSHOT.sh
浏览文件 @
ecb5595c
...
...
@@ -43,5 +43,6 @@ echo -e "\\tImport the project there using your favorite IDE (Import it as a mav
echo
-e
"
\\
tBuild a jar inside the directory using"
echo
-e
"
\\
t
\\
t mvn clean package"
echo
-e
"
\\
tYou will find the runnable jar in
$PACKAGE
/target"
echo
-e
"
\\
tConsult our website if you have any troubles: http://flink.apache.org/community.html#mailing-lists"
echo
-e
"
\\
n
\\
n"
flink-quickstart/quickstart-scala-SNAPSHOT.sh
浏览文件 @
ecb5595c
...
...
@@ -43,5 +43,6 @@ echo -e "\\tImport the project there using your favorite IDE (Import it as a mav
echo
-e
"
\\
tBuild a jar inside the directory using"
echo
-e
"
\\
t
\\
t mvn clean package"
echo
-e
"
\\
tYou will find the runnable jar in
$PACKAGE
/target"
echo
-e
"
\\
tConsult our website if you have any troubles: http://flink.apache.org/community.html#mailing-lists"
echo
-e
"
\\
n
\\
n"
flink-quickstart/quickstart-scala.sh
浏览文件 @
ecb5595c
...
...
@@ -42,6 +42,6 @@ echo -e "\\tImport the project there using your favorite IDE (Import it as a mav
echo
-e
"
\\
tBuild a jar inside the directory using"
echo
-e
"
\\
t
\\
t mvn clean package"
echo
-e
"
\\
tYou will find the runnable jar in
$PACKAGE
/target"
echo
-e
"
\\
tConsult our website if you have any troubles: http://flink.
incubator.
apache.org/community.html#mailing-lists"
echo
-e
"
\\
tConsult our website if you have any troubles: http://flink.apache.org/community.html#mailing-lists"
echo
-e
"
\\
n
\\
n"
flink-quickstart/quickstart.sh
浏览文件 @
ecb5595c
...
...
@@ -42,7 +42,7 @@ echo -e "\\tImport the project there using your favorite IDE (Import it as a mav
echo
-e
"
\\
tBuild a jar inside the directory using"
echo
-e
"
\\
t
\\
t mvn clean package"
echo
-e
"
\\
tYou will find the runnable jar in
$PACKAGE
/target"
echo
-e
"
\\
tConsult our website if you have any troubles: http://flink.
incubator.
apache.org/community.html#mailing-lists"
echo
-e
"
\\
tConsult our website if you have any troubles: http://flink.apache.org/community.html#mailing-lists"
echo
-e
"
\\
n
\\
n"
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录