Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
d5f2d7fa
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
d5f2d7fa
编写于
7月 14, 2014
作者:
G
gaborhermann
提交者:
Stephan Ewen
8月 18, 2014
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[streaming] StreamRecord created
上级
cf867f51
变更
5
隐藏空白更改
内联
并排
Showing
5 changed file
with
47 addition
and
36 deletion
+47
-36
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamRecord.java
...main/java/eu/stratosphere/streaming/api/StreamRecord.java
+34
-0
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamRecordProvider.java
...a/eu/stratosphere/streaming/api/StreamRecordProvider.java
+0
-21
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamTask.java
...c/main/java/eu/stratosphere/streaming/api/StreamTask.java
+4
-5
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/invokable/StreamInvokable.java
...stratosphere/streaming/api/invokable/StreamInvokable.java
+3
-4
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/test/wordcount/WordCount.java
...a/eu/stratosphere/streaming/test/wordcount/WordCount.java
+6
-6
未找到文件。
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamRecord.java
0 → 100644
浏览文件 @
d5f2d7fa
package
eu.stratosphere.streaming.api
;
import
java.util.Random
;
import
eu.stratosphere.types.LongValue
;
import
eu.stratosphere.types.Record
;
import
eu.stratosphere.types.Value
;
//TODO: refactor access modifiers
public
class
StreamRecord
{
private
Record
record
;
public
StreamRecord
(
Record
record
)
{
this
.
record
=
record
;
}
public
StreamRecord
addId
()
{
Random
rand
=
new
Random
();
record
.
addField
(
new
LongValue
(
rand
.
nextLong
()));
return
this
;
}
public
Long
popId
()
{
LongValue
id
=
new
LongValue
();
record
.
getFieldInto
(
record
.
getNumFields
()
-
1
,
id
);
record
.
removeField
(
record
.
getNumFields
()
-
1
);
return
id
.
getValue
();
}
public
Record
getRecord
()
{
return
record
;
}
}
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamRecordProvider.java
已删除
100644 → 0
浏览文件 @
cf867f51
package
eu.stratosphere.streaming.api
;
import
java.util.UUID
;
import
eu.stratosphere.types.LongValue
;
import
eu.stratosphere.types.Record
;
public
final
class
StreamRecordProvider
{
public
static
Long
addUUID
(
Record
rec
)
{
UUID
uuid
=
UUID
.
randomUUID
();
rec
.
addField
(
new
LongValue
(
uuid
.
getMostSignificantBits
()));
return
uuid
.
getLeastSignificantBits
();
}
public
static
Long
popUUID
(
Record
rec
)
{
LongValue
mostSignificantBits
=
new
LongValue
();
rec
.
getFieldInto
(
rec
.
getNumFields
()
-
1
,
mostSignificantBits
);
rec
.
removeField
(
rec
.
getNumFields
()
-
1
);
return
mostSignificantBits
.
getValue
();
}
}
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamTask.java
浏览文件 @
d5f2d7fa
...
...
@@ -127,12 +127,11 @@ public class StreamTask extends AbstractTask {
for
(
RecordReader
<
Record
>
input
:
inputs
)
{
if
(
input
.
hasNext
())
{
hasInput
=
true
;
Record
record
=
input
.
next
(
);
Long
id
=
StreamRecordProvider
.
popUUID
(
record
);
StreamRecord
streamRecord
=
new
StreamRecord
(
input
.
next
()
);
Long
id
=
streamRecord
.
popId
(
);
userFunction
.
invoke
(
record
);
//TODO: ack here
userFunction
.
invoke
(
streamRecord
.
getRecord
());
//TODO: ack here
}
}
}
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/invokable/StreamInvokable.java
浏览文件 @
d5f2d7fa
...
...
@@ -3,7 +3,7 @@ package eu.stratosphere.streaming.api.invokable;
import
java.util.List
;
import
eu.stratosphere.nephele.io.RecordWriter
;
import
eu.stratosphere.streaming.api.StreamRecord
Provider
;
import
eu.stratosphere.streaming.api.StreamRecord
;
import
eu.stratosphere.types.Record
;
public
abstract
class
StreamInvokable
{
...
...
@@ -17,9 +17,8 @@ public abstract class StreamInvokable {
public
final
void
emit
(
Record
record
)
{
for
(
RecordWriter
<
Record
>
output
:
outputs
)
{
try
{
//TODO: use this id for acking
Long
uuid
=
StreamRecordProvider
.
addUUID
(
record
);
output
.
emit
(
record
);
StreamRecord
streamRecord
=
new
StreamRecord
(
record
).
addId
();
output
.
emit
(
record
);
}
catch
(
Exception
e
)
{
System
.
out
.
println
(
"Emit error"
);
}
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/test/wordcount/WordCount.java
浏览文件 @
d5f2d7fa
...
...
@@ -26,16 +26,16 @@ public class WordCount extends TestBase2 {
@Override
public
JobGraph
getJobGraph
()
{
JobGraphBuilder
graphBuilder
=
new
JobGraphBuilder
(
"testGraph"
);
graphBuilder
.
setSource
(
"WorCountSource"
,
WordCountSource
.
class
);
graphBuilder
.
setTask
(
"WorCountSplitter"
,
WordCountSplitter
.
class
,
2
);
graphBuilder
.
setTask
(
"WorCountCounter"
,
WordCountCounter
.
class
,
2
);
graphBuilder
.
setSource
(
"Wor
d
CountSource"
,
WordCountSource
.
class
);
graphBuilder
.
setTask
(
"Wor
d
CountSplitter"
,
WordCountSplitter
.
class
,
2
);
graphBuilder
.
setTask
(
"Wor
d
CountCounter"
,
WordCountCounter
.
class
,
2
);
graphBuilder
.
setSink
(
"WordCountSink"
,
WordCountSink
.
class
);
graphBuilder
.
fieldsConnect
(
"Wor
CountSource"
,
"Wor
CountSplitter"
,
0
,
graphBuilder
.
fieldsConnect
(
"Wor
dCountSource"
,
"Word
CountSplitter"
,
0
,
StringValue
.
class
);
graphBuilder
.
fieldsConnect
(
"Wor
CountSplitter"
,
"Wor
CountCounter"
,
0
,
graphBuilder
.
fieldsConnect
(
"Wor
dCountSplitter"
,
"Word
CountCounter"
,
0
,
StringValue
.
class
);
graphBuilder
.
broadcastConnect
(
"WorCountCounter"
,
"WordCountSink"
);
graphBuilder
.
broadcastConnect
(
"Wor
d
CountCounter"
,
"WordCountSink"
);
return
graphBuilder
.
getJobGraph
();
}
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录