Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
26443e68
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
26443e68
编写于
7月 14, 2014
作者:
M
Márton Balassi
提交者:
Stephan Ewen
8月 18, 2014
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[streaming] UserFunction setter move to StreamComponent
上级
28ba7390
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
25 addition
and
16 deletion
+25
-16
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamComponentFactory.java
...eu/stratosphere/streaming/api/StreamComponentFactory.java
+23
-2
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamTask.java
...c/main/java/eu/stratosphere/streaming/api/StreamTask.java
+2
-14
未找到文件。
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamComponentFactory.java
浏览文件 @
26443e68
...
@@ -8,6 +8,8 @@ import eu.stratosphere.nephele.event.task.EventListener;
...
@@ -8,6 +8,8 @@ import eu.stratosphere.nephele.event.task.EventListener;
import
eu.stratosphere.nephele.io.ChannelSelector
;
import
eu.stratosphere.nephele.io.ChannelSelector
;
import
eu.stratosphere.nephele.io.RecordReader
;
import
eu.stratosphere.nephele.io.RecordReader
;
import
eu.stratosphere.nephele.io.RecordWriter
;
import
eu.stratosphere.nephele.io.RecordWriter
;
import
eu.stratosphere.streaming.api.invokable.DefaultTaskInvokable
;
import
eu.stratosphere.streaming.api.invokable.UserTaskInvokable
;
import
eu.stratosphere.streaming.partitioner.DefaultPartitioner
;
import
eu.stratosphere.streaming.partitioner.DefaultPartitioner
;
import
eu.stratosphere.streaming.partitioner.FieldsPartitioner
;
import
eu.stratosphere.streaming.partitioner.FieldsPartitioner
;
import
eu.stratosphere.types.Key
;
import
eu.stratosphere.types.Key
;
...
@@ -48,7 +50,7 @@ public final class StreamComponentFactory {
...
@@ -48,7 +50,7 @@ public final class StreamComponentFactory {
return
numberOfInputs
;
return
numberOfInputs
;
}
}
//for StreamTask
//
for StreamTask
public
static
int
setConfigOutputs
(
StreamTask
taskBase
,
public
static
int
setConfigOutputs
(
StreamTask
taskBase
,
Configuration
taskConfiguration
,
Configuration
taskConfiguration
,
List
<
RecordWriter
<
Record
>>
outputs
,
List
<
RecordWriter
<
Record
>>
outputs
,
...
@@ -65,7 +67,7 @@ public final class StreamComponentFactory {
...
@@ -65,7 +67,7 @@ public final class StreamComponentFactory {
}
}
return
numberOfOutputs
;
return
numberOfOutputs
;
}
}
// this function can be removed as duplication of the above function if
// this function can be removed as duplication of the above function if
// modification on kernel is allowed.
// modification on kernel is allowed.
// for StreamSource
// for StreamSource
...
@@ -86,6 +88,25 @@ public final class StreamComponentFactory {
...
@@ -86,6 +88,25 @@ public final class StreamComponentFactory {
return
numberOfOutputs
;
return
numberOfOutputs
;
}
}
public
static
UserTaskInvokable
setUserFunction
(
Configuration
taskConfiguration
,
List
<
RecordWriter
<
Record
>>
outputs
,
String
instanceID
,
Map
<
String
,
StreamRecord
>
recordBuffer
)
{
Class
<?
extends
UserTaskInvokable
>
userFunctionClass
=
taskConfiguration
.
getClass
(
"userfunction"
,
DefaultTaskInvokable
.
class
,
UserTaskInvokable
.
class
);
UserTaskInvokable
userFunction
=
null
;
try
{
userFunction
=
userFunctionClass
.
newInstance
();
userFunction
.
declareOutputs
(
outputs
,
instanceID
,
recordBuffer
);
}
catch
(
Exception
e
)
{
}
return
userFunction
;
}
public
static
void
setPartitioner
(
Configuration
taskConfiguration
,
public
static
void
setPartitioner
(
Configuration
taskConfiguration
,
int
nrOutput
,
List
<
ChannelSelector
<
Record
>>
partitioners
)
{
int
nrOutput
,
List
<
ChannelSelector
<
Record
>>
partitioners
)
{
Class
<?
extends
ChannelSelector
<
Record
>>
partitioner
=
taskConfiguration
Class
<?
extends
ChannelSelector
<
Record
>>
partitioner
=
taskConfiguration
...
...
flink-addons/flink-streaming/src/main/java/eu/stratosphere/streaming/api/StreamTask.java
浏览文件 @
26443e68
...
@@ -25,7 +25,6 @@ import eu.stratosphere.nephele.io.ChannelSelector;
...
@@ -25,7 +25,6 @@ import eu.stratosphere.nephele.io.ChannelSelector;
import
eu.stratosphere.nephele.io.RecordReader
;
import
eu.stratosphere.nephele.io.RecordReader
;
import
eu.stratosphere.nephele.io.RecordWriter
;
import
eu.stratosphere.nephele.io.RecordWriter
;
import
eu.stratosphere.nephele.template.AbstractTask
;
import
eu.stratosphere.nephele.template.AbstractTask
;
import
eu.stratosphere.streaming.api.invokable.DefaultTaskInvokable
;
import
eu.stratosphere.streaming.api.invokable.UserTaskInvokable
;
import
eu.stratosphere.streaming.api.invokable.UserTaskInvokable
;
import
eu.stratosphere.types.Record
;
import
eu.stratosphere.types.Record
;
...
@@ -56,18 +55,6 @@ public class StreamTask extends AbstractTask {
...
@@ -56,18 +55,6 @@ public class StreamTask extends AbstractTask {
recordBuffer
=
new
TreeMap
<
String
,
StreamRecord
>();
recordBuffer
=
new
TreeMap
<
String
,
StreamRecord
>();
}
}
public
void
setUserFunction
(
Configuration
taskConfiguration
)
{
Class
<?
extends
UserTaskInvokable
>
userFunctionClass
=
taskConfiguration
.
getClass
(
"userfunction"
,
DefaultTaskInvokable
.
class
,
UserTaskInvokable
.
class
);
try
{
userFunction
=
userFunctionClass
.
newInstance
();
userFunction
.
declareOutputs
(
outputs
,
taskInstanceID
,
recordBuffer
);
}
catch
(
Exception
e
)
{
}
}
@Override
@Override
public
void
registerInputOutput
()
{
public
void
registerInputOutput
()
{
Configuration
taskConfiguration
=
getTaskConfiguration
();
Configuration
taskConfiguration
=
getTaskConfiguration
();
...
@@ -77,7 +64,8 @@ public class StreamTask extends AbstractTask {
...
@@ -77,7 +64,8 @@ public class StreamTask extends AbstractTask {
numberOfOutputs
=
StreamComponentFactory
.
setConfigOutputs
(
this
,
numberOfOutputs
=
StreamComponentFactory
.
setConfigOutputs
(
this
,
taskConfiguration
,
outputs
,
partitioners
);
taskConfiguration
,
outputs
,
partitioners
);
setUserFunction
(
taskConfiguration
);
userFunction
=
StreamComponentFactory
.
setUserFunction
(
taskConfiguration
,
outputs
,
taskInstanceID
,
recordBuffer
);
StreamComponentFactory
.
setAckListener
(
recordBuffer
,
taskInstanceID
,
StreamComponentFactory
.
setAckListener
(
recordBuffer
,
taskInstanceID
,
outputs
);
outputs
);
}
}
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录