Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
88d7305a
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
88d7305a
编写于
8月 10, 2013
作者:
R
Robert Metzger
提交者:
StephanEwen
8月 16, 2013
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
added test case (without fix) for a pact compiler bug with ALL_GROUP-reducer.
上级
d5f86db2
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
47 addition
and
3 deletion
+47
-3
nephele/nephele-common/src/main/java/eu/stratosphere/nephele/fs/FileSystem.java
.../src/main/java/eu/stratosphere/nephele/fs/FileSystem.java
+1
-1
pact/pact-common/src/main/java/eu/stratosphere/pact/common/io/DelimitedInputFormat.java
.../eu/stratosphere/pact/common/io/DelimitedInputFormat.java
+2
-2
pact/pact-compiler/src/test/java/eu/stratosphere/pact/compiler/ReduceCompilerBugTest.java
.../eu/stratosphere/pact/compiler/ReduceCompilerBugTest.java
+44
-0
未找到文件。
nephele/nephele-common/src/main/java/eu/stratosphere/nephele/fs/FileSystem.java
浏览文件 @
88d7305a
...
@@ -190,7 +190,7 @@ public abstract class FileSystem {
...
@@ -190,7 +190,7 @@ public abstract class FileSystem {
synchronized
(
SYNCHRONIZATION_OBJECT
)
{
synchronized
(
SYNCHRONIZATION_OBJECT
)
{
if
(
uri
.
getScheme
()
==
null
)
{
if
(
uri
.
getScheme
()
==
null
)
{
throw
new
IOException
(
"FileSystem: Scheme is null"
);
throw
new
IOException
(
"FileSystem: Scheme is null
. file:// or hdfs:// are schemes.
"
);
}
}
final
FSKey
key
=
new
FSKey
(
uri
.
getScheme
(),
uri
.
getAuthority
());
final
FSKey
key
=
new
FSKey
(
uri
.
getScheme
(),
uri
.
getAuthority
());
...
...
pact/pact-common/src/main/java/eu/stratosphere/pact/common/io/DelimitedInputFormat.java
浏览文件 @
88d7305a
...
@@ -72,7 +72,7 @@ public abstract class DelimitedInputFormat extends FileInputFormat {
...
@@ -72,7 +72,7 @@ public abstract class DelimitedInputFormat extends FileInputFormat {
*/
*/
private
static
int
MAX_SAMPLE_LEN
;
private
static
int
MAX_SAMPLE_LEN
;
protected
static
final
void
loadGlo
ab
lConfigParams
()
{
protected
static
final
void
loadGlo
ba
lConfigParams
()
{
int
maxSamples
=
GlobalConfiguration
.
getInteger
(
PactConfigConstants
.
DELIMITED_FORMAT_MAX_LINE_SAMPLES_KEY
,
int
maxSamples
=
GlobalConfiguration
.
getInteger
(
PactConfigConstants
.
DELIMITED_FORMAT_MAX_LINE_SAMPLES_KEY
,
PactConfigConstants
.
DEFAULT_DELIMITED_FORMAT_MAX_LINE_SAMPLES
);
PactConfigConstants
.
DEFAULT_DELIMITED_FORMAT_MAX_LINE_SAMPLES
);
int
minSamples
=
GlobalConfiguration
.
getInteger
(
PactConfigConstants
.
DELIMITED_FORMAT_MIN_LINE_SAMPLES_KEY
,
int
minSamples
=
GlobalConfiguration
.
getInteger
(
PactConfigConstants
.
DELIMITED_FORMAT_MIN_LINE_SAMPLES_KEY
,
...
@@ -111,7 +111,7 @@ public abstract class DelimitedInputFormat extends FileInputFormat {
...
@@ -111,7 +111,7 @@ public abstract class DelimitedInputFormat extends FileInputFormat {
MAX_SAMPLE_LEN
=
maxLen
;
MAX_SAMPLE_LEN
=
maxLen
;
}
}
static
{
loadGlo
ab
lConfigParams
();
}
static
{
loadGlo
ba
lConfigParams
();
}
// ------------------------------------- Config Keys ------------------------------------------
// ------------------------------------- Config Keys ------------------------------------------
...
...
pact/pact-compiler/src/test/java/eu/stratosphere/pact/compiler/ReduceCompilerBugTest.java
0 → 100644
浏览文件 @
88d7305a
package
eu.stratosphere.pact.compiler
;
import
static
org
.
junit
.
Assert
.*;
import
org.junit.Test
;
import
eu.stratosphere.pact.common.contract.FileDataSink
;
import
eu.stratosphere.pact.common.contract.FileDataSource
;
import
eu.stratosphere.pact.common.contract.ReduceContract
;
import
eu.stratosphere.pact.common.plan.Plan
;
import
eu.stratosphere.pact.compiler.plan.candidate.OptimizedPlan
;
import
eu.stratosphere.pact.compiler.plantranslate.NepheleJobGraphGenerator
;
import
eu.stratosphere.pact.compiler.util.DummyInputFormat
;
import
eu.stratosphere.pact.compiler.util.DummyOutputFormat
;
import
eu.stratosphere.pact.compiler.util.IdentityReduce
;
/**
* This test case has been created to validate a bug that occurred when
* the ReduceContract was used without a grouping key.
*/
public
class
ReduceCompilerBugTest
extends
CompilerTestBase
{
@Test
public
void
testReduce
()
{
// construct the plan
FileDataSource
source
=
new
FileDataSource
(
DummyInputFormat
.
class
,
IN_FILE
,
"Source"
);
ReduceContract
reduce1
=
ReduceContract
.
builder
(
IdentityReduce
.
class
).
name
(
"Reduce1"
).
input
(
source
).
build
();
FileDataSink
sink
=
new
FileDataSink
(
DummyOutputFormat
.
class
,
OUT_FILE
,
"Sink"
);
sink
.
setInput
(
reduce1
);
Plan
plan
=
new
Plan
(
sink
,
"Test Temp Task"
);
plan
.
setDefaultParallelism
(
DEFAULT_PARALLELISM
);
try
{
OptimizedPlan
oPlan
=
compileNoStats
(
plan
);
NepheleJobGraphGenerator
jobGen
=
new
NepheleJobGraphGenerator
();
jobGen
.
compileJobGraph
(
oPlan
);
}
catch
(
CompilerException
ce
)
{
ce
.
printStackTrace
();
fail
(
"The pact compiler is unable to compile this plan correctly"
);
}
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录