Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
bd81e2c3
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
bd81e2c3
编写于
7月 14, 2014
作者:
G
gaborhermann
提交者:
Stephan Ewen
8月 18, 2014
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[streaming] StreamRecord copy quickfix
上级
700b38d7
变更
9
隐藏空白更改
内联
并排
Showing
9 changed file
with
42 addition
and
20 deletion
+42
-20
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/AckEvent.java
...src/main/java/eu/stratosphere/streaming/api/AckEvent.java
+13
-11
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamRecord.java
...main/java/eu/stratosphere/streaming/api/StreamRecord.java
+2
-0
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/invokable/UserSourceInvokable.java
...tosphere/streaming/api/invokable/UserSourceInvokable.java
+2
-0
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/invokable/UserTaskInvokable.java
...ratosphere/streaming/api/invokable/UserTaskInvokable.java
+1
-0
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/streamcomponent/StreamComponentHelper.java
.../streaming/api/streamcomponent/StreamComponentHelper.java
+0
-1
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/streamcomponent/StreamInvokable.java
...sphere/streaming/api/streamcomponent/StreamInvokable.java
+5
-5
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/streamcomponent/StreamSource.java
...atosphere/streaming/api/streamcomponent/StreamSource.java
+2
-2
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/test/wordcount/WordCount.java
...a/eu/stratosphere/streaming/test/wordcount/WordCount.java
+14
-0
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/test/wordcount/WordCountLocal.java
...stratosphere/streaming/test/wordcount/WordCountLocal.java
+3
-1
未找到文件。
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/AckEvent.java
浏览文件 @
bd81e2c3
...
...
@@ -23,30 +23,32 @@ import org.apache.commons.logging.Log;
import
org.apache.commons.logging.LogFactory
;
import
eu.stratosphere.nephele.event.task.AbstractTaskEvent
;
/**TaskEvent for sending record acknowledgements to the input's fault tolerance buffer
/**
* TaskEvent for sending record acknowledgements to the input's fault tolerance
* buffer
*
*
*
*/
public
class
AckEvent
extends
AbstractTaskEvent
{
private
static
final
Log
log
=
LogFactory
.
getLog
(
AckEvent
.
class
);
private
String
recordId
;
/**Creates a new event to acknowledge the record with the given ID
/**
* Creates a new event to acknowledge the record with the given ID
*
* @param recordId
* ID of the record to be acknowledged
*
ID of the record to be acknowledged
*/
public
AckEvent
(
String
recordId
)
{
setRecordId
(
recordId
);
if
(
log
.
isDebugEnabled
())
{
log
.
debug
(
"Ack sent "
+
recordId
);
}
// System.out.println("Ack sent " + recordId);
// System.out.println("---------------------");
}
@Override
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamRecord.java
浏览文件 @
bd81e2c3
...
...
@@ -248,6 +248,8 @@ public class StreamRecord implements IOReadableWritable, Serializable {
*/
public
StreamRecord
copy
()
{
StreamRecord
copiedRecord
=
new
StreamRecord
(
this
.
numOfFields
,
this
.
numOfRecords
);
copiedRecord
.
uid
=
this
.
uid
;
for
(
Value
[]
record
:
recordBatch
)
{
copiedRecord
.
recordBatch
.
add
(
record
);
}
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/invokable/UserSourceInvokable.java
浏览文件 @
bd81e2c3
...
...
@@ -15,6 +15,8 @@
package
eu.stratosphere.streaming.api.invokable
;
import
eu.stratosphere.streaming.api.streamcomponent.StreamInvokable
;
public
abstract
class
UserSourceInvokable
extends
StreamInvokable
{
public
void
invoke
()
throws
Exception
{
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/invokable/UserTaskInvokable.java
浏览文件 @
bd81e2c3
...
...
@@ -16,6 +16,7 @@
package
eu.stratosphere.streaming.api.invokable
;
import
eu.stratosphere.streaming.api.StreamRecord
;
import
eu.stratosphere.streaming.api.streamcomponent.StreamInvokable
;
public
abstract
class
UserTaskInvokable
extends
StreamInvokable
{
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/streamcomponent/StreamComponentHelper.java
浏览文件 @
bd81e2c3
...
...
@@ -19,7 +19,6 @@ import eu.stratosphere.streaming.api.FaultToleranceBuffer;
import
eu.stratosphere.streaming.api.StreamRecord
;
import
eu.stratosphere.streaming.api.invokable.DefaultSinkInvokable
;
import
eu.stratosphere.streaming.api.invokable.DefaultTaskInvokable
;
import
eu.stratosphere.streaming.api.invokable.StreamInvokable
;
import
eu.stratosphere.streaming.api.invokable.UserSinkInvokable
;
import
eu.stratosphere.streaming.partitioner.DefaultPartitioner
;
import
eu.stratosphere.streaming.partitioner.FieldsPartitioner
;
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/
invokable
/StreamInvokable.java
→
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/
streamcomponent
/StreamInvokable.java
浏览文件 @
bd81e2c3
package
eu.stratosphere.streaming.api.
invokable
;
package
eu.stratosphere.streaming.api.
streamcomponent
;
import
java.util.List
;
...
...
@@ -29,10 +29,10 @@ public abstract class StreamInvokable {
try
{
output
.
emit
(
record
);
System
.
out
.
println
(
this
.
getClass
().
getName
());
System
.
out
.
println
(
"Emitted "
+
record
.
getId
()
+
"-"
+
record
.
toString
());
System
.
out
.
println
(
"---------------------"
);
//
System.out.println(this.getClass().getName());
//
System.out.println("Emitted " + record.getId() + "-"
//
+ record.toString());
//
System.out.println("---------------------");
}
catch
(
Exception
e
)
{
System
.
out
.
println
(
"Emit error: "
+
e
.
getMessage
());
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/streamcomponent/StreamSource.java
浏览文件 @
bd81e2c3
...
...
@@ -80,8 +80,8 @@ public class StreamSource extends AbstractInputTask<RandIS> {
public
void
invoke
()
throws
Exception
{
userFunction
.
invoke
();
System
.
out
.
println
(
this
.
getClass
().
getName
()
+
"-"
+
sourceInstanceID
);
System
.
out
.
println
(
"---------------------"
);
//
System.out.println(this.getClass().getName() + "-" + sourceInstanceID);
//
System.out.println("---------------------");
}
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/test/wordcount/WordCount.java
浏览文件 @
bd81e2c3
...
...
@@ -15,6 +15,11 @@
package
eu.stratosphere.streaming.test.wordcount
;
import
org.apache.log4j.ConsoleAppender
;
import
org.apache.log4j.Level
;
import
org.apache.log4j.Logger
;
import
org.apache.log4j.PatternLayout
;
import
eu.stratosphere.nephele.jobgraph.JobGraph
;
import
eu.stratosphere.streaming.api.JobGraphBuilder
;
import
eu.stratosphere.test.util.TestBase2
;
...
...
@@ -24,6 +29,15 @@ public class WordCount extends TestBase2 {
@Override
public
JobGraph
getJobGraph
()
{
Logger
root
=
Logger
.
getRootLogger
();
root
.
removeAllAppenders
();
PatternLayout
layout
=
new
PatternLayout
(
"%d{HH:mm:ss,SSS} %-5p %-60c %x - %m%n"
);
ConsoleAppender
appender
=
new
ConsoleAppender
(
layout
,
"System.err"
);
root
.
addAppender
(
appender
);
root
.
setLevel
(
Level
.
DEBUG
);
JobGraphBuilder
graphBuilder
=
new
JobGraphBuilder
(
"testGraph"
);
graphBuilder
.
setSource
(
"WordCountSource"
,
WordCountDummySource
.
class
);
graphBuilder
.
setTask
(
"WordCountSplitter"
,
WordCountSplitter
.
class
,
2
);
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/test/wordcount/WordCountLocal.java
浏览文件 @
bd81e2c3
...
...
@@ -16,6 +16,7 @@
package
eu.stratosphere.streaming.test.wordcount
;
import
org.apache.hadoop.fs.FileSystem
;
import
org.apache.log4j.BasicConfigurator
;
import
org.apache.log4j.Level
;
import
org.junit.Assert
;
...
...
@@ -106,7 +107,7 @@ public class WordCountLocal {
protected
JobGraph
getJobGraph
()
throws
Exception
{
JobGraphBuilder
graphBuilder
=
new
JobGraphBuilder
(
"testGraph"
);
graphBuilder
.
setSource
(
"WordCountSource"
,
WordCountSource
.
class
);
graphBuilder
.
setSource
(
"WordCountSource"
,
WordCount
Dummy
Source
.
class
);
graphBuilder
.
setTask
(
"WordCountSplitter"
,
WordCountSplitter
.
class
,
2
);
graphBuilder
.
setTask
(
"WordCountCounter"
,
WordCountCounter
.
class
,
2
);
graphBuilder
.
setSink
(
"WordCountSink"
,
WordCountSink
.
class
);
...
...
@@ -123,6 +124,7 @@ public class WordCountLocal {
public
static
void
main
(
String
[]
args
){
WordCountLocal
wC
=
new
WordCountLocal
();
BasicConfigurator
.
configure
();
try
{
wC
.
startCluster
();
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录