Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
fd280981
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
fd280981
编写于
9月 16, 2014
作者:
A
Aljoscha Krettek
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[scala] Reactivate DeltaIterationSanityCheckTest
上级
31ed0c4c
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
169 addition
and
192 deletion
+169
-192
flink-scala/src/test/scala/org/apache/flink/api/scala/DeltaIterationSanityCheckTest.scala
...pache/flink/api/scala/DeltaIterationSanityCheckTest.scala
+169
-192
未找到文件。
flink-scala/src/test/scala/org/apache/flink/api/scala/DeltaIterationSanityCheckTest.scala
浏览文件 @
fd280981
///**
// * Licensed to the Apache Software Foundation (ASF) under one
// * or more contributor license agreements. See the NOTICE file
// * distributed with this work for additional information
// * regarding copyright ownership. The ASF licenses this file
// * to you under the Apache License, Version 2.0 (the
// * "License"); you may not use this file except in compliance
// * with the License. You may obtain a copy of the License at
// *
// * http://www.apache.org/licenses/LICENSE-2.0
// *
// * Unless required by applicable law or agreed to in writing, software
// * distributed under the License is distributed on an "AS IS" BASIS,
// * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// * See the License for the specific language governing permissions and
// * limitations under the License.
// */
//
//package org.apache.flink.api.scala
//
//import org.junit.Test
//import org.apache.flink.api.common.InvalidProgramException
//
//import org.apache.flink.api.scala._
//import org.apache.flink.api.scala.operators._
//import org.scalatest.junit.AssertionsForJUnit
//
//// Verify that the sanity checking in delta iterations works. We just
//// have a dummy job that is not meant to be executed. Only verify that
//// the join/coGroup inside the iteration is checked.
//class DeltaIterationSanityCheckTest extends Serializable {
//
// @Test
// def testCorrectJoinWithSolution1 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = s join ws where {_._1} isEqualTo {_._1} map { (l, r) => l }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._1}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//
// @Test
// def testCorrectJoinWithSolution2 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = ws join s where {_._1} isEqualTo {_._1} map { (l, r) => l }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._1}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//
// @Test(expected = classOf[InvalidProgramException])
// def testIncorrectJoinWithSolution1 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = s join ws where {_._2} isEqualTo {_._2} map { (l, r) => l }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._1}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//
// @Test(expected = classOf[InvalidProgramException])
// def testIncorrectJoinWithSolution2 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = ws join s where {_._2} isEqualTo {_._2} map { (l, r) => l }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._1}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//
// @Test(expected = classOf[InvalidProgramException])
// def testIncorrectJoinWithSolution3 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = s join ws where {_._1} isEqualTo {_._1} map { (l, r) => l }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._2}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//
// @Test
// def testCorrectCoGroupWithSolution1 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = s cogroup ws where {_._1} isEqualTo {_._1} map { (l, r) => l.next() }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._1}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//
// @Test
// def testCorrectCoGroupWithSolution2 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = ws cogroup s where {_._1} isEqualTo {_._1} map { (l, r) => l.next() }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._1}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//
// @Test(expected = classOf[InvalidProgramException])
// def testIncorrectCoGroupWithSolution1 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = s cogroup ws where {_._2} isEqualTo {_._2} map { (l, r) => l.next() }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._1}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//
// @Test(expected = classOf[InvalidProgramException])
// def testIncorrectCoGroupWithSolution2 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = ws cogroup s where {_._2} isEqualTo {_._2} map { (l, r) => l.next() }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._1}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//
// @Test(expected = classOf[InvalidProgramException])
// def testIncorrectCoGroupWithSolution3 {
// val solutionInput = CollectionDataSource(Array((1, "1")))
// val worksetInput = CollectionDataSource(Array((2, "2")))
//
// def step(s: DataSetOLD[(Int, String)], ws: DataSetOLD[(Int, String)]) = {
// val result = s cogroup ws where {_._1} isEqualTo {_._1} map { (l, r) => l.next() }
// (result, ws)
// }
// val iteration = solutionInput.iterateWithDelta(worksetInput, {_._2}, step, 10)
//
// val output = iteration.write("/dummy", CsvOutputFormat())
//
// val plan = new ScalaPlan(Seq(output))
// }
//}
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org.apache.flink.api.scala
import
org.junit.Test
import
org.apache.flink.api.common.InvalidProgramException
import
org.apache.flink.api.scala._
import
org.scalatest.junit.AssertionsForJUnit
// Verify that the sanity checking in delta iterations works. We just
// have a dummy job that is not meant to be executed. Only verify that
// the join/coGroup inside the iteration is checked.
class
DeltaIterationSanityCheckTest
extends
Serializable
{
@Test
def
testCorrectJoinWithSolution1
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_1"
))
{
(
s
,
ws
)
=>
val
result
=
s
.
join
(
ws
).
where
(
"_1"
).
equalTo
(
"_1"
)
{
(
l
,
r
)
=>
l
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
@Test
def
testCorrectJoinWithSolution2
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_1"
))
{
(
s
,
ws
)
=>
val
result
=
ws
.
join
(
s
).
where
(
"_1"
).
equalTo
(
"_1"
)
{
(
l
,
r
)
=>
l
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
@Test
(
expected
=
classOf
[
InvalidProgramException
])
def
testIncorrectJoinWithSolution1
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_1"
))
{
(
s
,
ws
)
=>
val
result
=
s
.
join
(
ws
).
where
(
"_2"
).
equalTo
(
"_2"
)
{
(
l
,
r
)
=>
l
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
@Test
(
expected
=
classOf
[
InvalidProgramException
])
def
testIncorrectJoinWithSolution2
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_1"
))
{
(
s
,
ws
)
=>
val
result
=
ws
.
join
(
s
).
where
(
"_2"
).
equalTo
(
"_2"
)
{
(
l
,
r
)
=>
l
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
@Test
(
expected
=
classOf
[
InvalidProgramException
])
def
testIncorrectJoinWithSolution3
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_2"
))
{
(
s
,
ws
)
=>
val
result
=
ws
.
join
(
s
).
where
(
"_1"
).
equalTo
(
"_1"
)
{
(
l
,
r
)
=>
l
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
@Test
def
testCorrectCoGroupWithSolution1
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_1"
))
{
(
s
,
ws
)
=>
val
result
=
s
.
coGroup
(
ws
).
where
(
"_1"
).
equalTo
(
"_1"
)
{
(
l
,
r
)
=>
l
.
min
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
@Test
def
testCorrectCoGroupWithSolution2
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_1"
))
{
(
s
,
ws
)
=>
val
result
=
ws
.
coGroup
(
s
).
where
(
"_1"
).
equalTo
(
"_1"
)
{
(
l
,
r
)
=>
l
.
min
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
@Test
(
expected
=
classOf
[
InvalidProgramException
])
def
testIncorrectCoGroupWithSolution1
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_1"
))
{
(
s
,
ws
)
=>
val
result
=
s
.
coGroup
(
ws
).
where
(
"_2"
).
equalTo
(
"_2"
)
{
(
l
,
r
)
=>
l
.
min
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
@Test
(
expected
=
classOf
[
InvalidProgramException
])
def
testIncorrectCoGroupWithSolution2
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_1"
))
{
(
s
,
ws
)
=>
val
result
=
ws
.
coGroup
(
s
).
where
(
"_2"
).
equalTo
(
"_2"
)
{
(
l
,
r
)
=>
l
.
min
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
@Test
(
expected
=
classOf
[
InvalidProgramException
])
def
testIncorrectCoGroupWithSolution3
()
:
Unit
=
{
val
env
=
ExecutionEnvironment
.
getExecutionEnvironment
val
solutionInput
=
env
.
fromElements
((
1
,
"1"
))
val
worksetInput
=
env
.
fromElements
((
2
,
"2"
))
val
iteration
=
solutionInput
.
iterateDelta
(
worksetInput
,
10
,
Array
(
"_2"
))
{
(
s
,
ws
)
=>
val
result
=
ws
.
coGroup
(
s
).
where
(
"_1"
).
equalTo
(
"_1"
)
{
(
l
,
r
)
=>
l
.
min
}
(
result
,
ws
)
}
val
output
=
iteration
.
print
()
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录