Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
35438ec2
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
35438ec2
编写于
6月 25, 2014
作者:
S
Stephan Ewen
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
InputVertices do not require an input format.
上级
ef623e9b
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
14 addition
and
64 deletion
+14
-64
stratosphere-runtime/src/main/java/eu/stratosphere/nephele/executiongraph/ExecutionGroupVertex.java
...tosphere/nephele/executiongraph/ExecutionGroupVertex.java
+3
-3
stratosphere-runtime/src/main/java/eu/stratosphere/nephele/jobgraph/JobInputVertex.java
...java/eu/stratosphere/nephele/jobgraph/JobInputVertex.java
+3
-3
stratosphere-tests/src/test/java/eu/stratosphere/test/runtime/NetworkStackThroughput.java
.../eu/stratosphere/test/runtime/NetworkStackThroughput.java
+8
-58
未找到文件。
stratosphere-runtime/src/main/java/eu/stratosphere/nephele/executiongraph/ExecutionGroupVertex.java
浏览文件 @
35438ec2
...
...
@@ -429,8 +429,9 @@ public final class ExecutionGroupVertex {
*
* @param inputSplitType Input split type class
*/
public
void
setInputSplitType
(
final
Class
<?
extends
InputSplit
>
inputSplitType
)
{
this
.
inputSplitType
=
inputSplitType
;
}
public
void
setInputSplitType
(
final
Class
<?
extends
InputSplit
>
inputSplitType
)
{
this
.
inputSplitType
=
inputSplitType
;
}
/**
* Returns the input splits assigned to this group vertex.
...
...
@@ -438,7 +439,6 @@ public final class ExecutionGroupVertex {
* @return the input splits, possibly <code>null</code> if the group vertex does not represent an input vertex
*/
public
InputSplit
[]
getInputSplits
()
{
return
this
.
inputSplits
;
}
...
...
stratosphere-runtime/src/main/java/eu/stratosphere/nephele/jobgraph/JobInputVertex.java
浏览文件 @
35438ec2
...
...
@@ -72,8 +72,8 @@ public class JobInputVertex extends AbstractJobInputVertex {
*/
@Override
public
Class
<?
extends
InputSplit
>
getInputSplitType
()
{
if
(
inputFormat
==
null
){
throw
new
RuntimeException
(
"No input format has been set for job vertex: "
+
this
.
getID
())
;
if
(
inputFormat
==
null
){
return
InputSplit
.
class
;
}
return
inputFormat
.
getInputSplitType
();
...
...
@@ -89,7 +89,7 @@ public class JobInputVertex extends AbstractJobInputVertex {
@Override
public
InputSplit
[]
getInputSplits
(
int
minNumSplits
)
throws
IOException
{
if
(
inputFormat
==
null
){
throw
new
RuntimeException
(
"No input format has been set for job vertex: "
+
this
.
getID
())
;
return
null
;
}
return
inputFormat
.
createInputSplits
(
minNumSplits
);
...
...
stratosphere-tests/src/test/java/eu/stratosphere/test/runtime/NetworkStackThroughput.java
浏览文件 @
35438ec2
...
...
@@ -13,12 +13,16 @@
package
eu.stratosphere.test.runtime
;
import
eu.stratosphere.api.common.io.GenericInputFormat
;
import
eu.stratosphere.api.common.io.OutputFormat
;
import
eu.stratosphere.api.common.operators.util.UserCodeObjectWrapper
;
import
java.io.DataInput
;
import
java.io.DataOutput
;
import
java.io.IOException
;
import
org.apache.commons.logging.Log
;
import
org.apache.commons.logging.LogFactory
;
import
org.junit.After
;
import
eu.stratosphere.configuration.Configuration
;
import
eu.stratosphere.core.io.IOReadableWritable
;
import
eu.stratosphere.core.io.InputSplit
;
import
eu.stratosphere.nephele.jobgraph.DistributionPattern
;
import
eu.stratosphere.nephele.jobgraph.JobGraph
;
import
eu.stratosphere.nephele.jobgraph.JobGraphDefinitionException
;
...
...
@@ -26,20 +30,11 @@ import eu.stratosphere.nephele.jobgraph.JobInputVertex;
import
eu.stratosphere.nephele.jobgraph.JobOutputVertex
;
import
eu.stratosphere.nephele.jobgraph.JobTaskVertex
;
import
eu.stratosphere.nephele.template.AbstractInvokable
;
import
eu.stratosphere.pact.runtime.task.util.TaskConfig
;
import
eu.stratosphere.runtime.io.api.RecordReader
;
import
eu.stratosphere.runtime.io.api.RecordWriter
;
import
eu.stratosphere.runtime.io.channels.ChannelType
;
import
eu.stratosphere.test.util.RecordAPITestBase
;
import
eu.stratosphere.types.Record
;
import
eu.stratosphere.util.LogUtils
;
import
org.apache.commons.logging.Log
;
import
org.apache.commons.logging.LogFactory
;
import
org.junit.After
;
import
java.io.DataInput
;
import
java.io.DataOutput
;
import
java.io.IOException
;
public
class
NetworkStackThroughput
{
...
...
@@ -109,9 +104,6 @@ public class NetworkStackThroughput {
producer
.
getConfiguration
().
setInteger
(
DATA_VOLUME_GB_CONFIG_KEY
,
dataVolumeGb
);
producer
.
getConfiguration
().
setBoolean
(
IS_SLOW_SENDER_CONFIG_KEY
,
isSlowSender
);
TaskConfig
inputConfig
=
new
TaskConfig
(
producer
.
getConfiguration
());
inputConfig
.
setStubWrapper
(
new
UserCodeObjectWrapper
<
Object
>(
new
DummyInputFormat
()));
JobTaskVertex
forwarder
=
null
;
if
(
useForwarder
)
{
forwarder
=
new
JobTaskVertex
(
"Speed Test Forwarder"
,
jobGraph
);
...
...
@@ -124,9 +116,6 @@ public class NetworkStackThroughput {
consumer
.
setNumberOfSubtasks
(
numSubtasks
);
consumer
.
getConfiguration
().
setBoolean
(
IS_SLOW_RECEIVER_CONFIG_KEY
,
isSlowReceiver
);
TaskConfig
outputConfig
=
new
TaskConfig
(
consumer
.
getConfiguration
());
outputConfig
.
setStubWrapper
(
new
UserCodeObjectWrapper
<
Object
>(
new
DummyOutputFormat
()));
if
(
useForwarder
)
{
producer
.
connectTo
(
forwarder
,
ChannelType
.
NETWORK
,
DistributionPattern
.
BIPARTITE
);
forwarder
.
connectTo
(
consumer
,
ChannelType
.
NETWORK
,
DistributionPattern
.
BIPARTITE
);
...
...
@@ -269,45 +258,6 @@ public class NetworkStackThroughput {
}
}
public
static
final
class
DummyInputFormat
extends
GenericInputFormat
{
private
static
final
long
serialVersionUID
=
6891640958330871924L
;
@Override
public
void
open
(
InputSplit
split
)
throws
IOException
{
}
@Override
public
boolean
reachedEnd
()
throws
IOException
{
return
false
;
}
@Override
public
Object
nextRecord
(
Object
reuse
)
throws
IOException
{
return
null
;
}
}
public
static
final
class
DummyOutputFormat
implements
OutputFormat
<
Record
>
{
@Override
public
void
configure
(
Configuration
parameters
)
{
}
@Override
public
void
open
(
int
taskNumber
,
int
numTasks
)
{
}
@Override
public
void
writeRecord
(
Record
record
)
{
}
@Override
public
void
close
()
{
}
}
// ------------------------------------------------------------------------
public
void
testThroughput
()
throws
Exception
{
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录