Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
87749ca4
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
87749ca4
编写于
11月 30, 2017
作者:
G
gyao
提交者:
Till Rohrmann
12月 13, 2017
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[hotfix][tests] Extract SubmittedJobGraphStore implementation from JobManagerHARecoveryTest
上级
01d0d256
变更
2
显示空白变更内容
内联
并排
Showing
2 changed file
with
90 addition
and
47 deletion
+90
-47
flink-runtime/src/test/java/org/apache/flink/runtime/jobmanager/JobManagerHARecoveryTest.java
...he/flink/runtime/jobmanager/JobManagerHARecoveryTest.java
+6
-47
flink-runtime/src/test/java/org/apache/flink/runtime/testutils/InMemorySubmittedJobGraphStore.java
...ink/runtime/testutils/InMemorySubmittedJobGraphStore.java
+84
-0
未找到文件。
flink-runtime/src/test/java/org/apache/flink/runtime/jobmanager/JobManagerHARecoveryTest.java
浏览文件 @
87749ca4
...
...
@@ -74,6 +74,7 @@ import org.apache.flink.runtime.testingUtils.TestingMessages;
import
org.apache.flink.runtime.testingUtils.TestingTaskManager
;
import
org.apache.flink.runtime.testingUtils.TestingTaskManagerMessages
;
import
org.apache.flink.runtime.testingUtils.TestingUtils
;
import
org.apache.flink.runtime.testutils.InMemorySubmittedJobGraphStore
;
import
org.apache.flink.runtime.testutils.RecoverableCompletedCheckpointStore
;
import
org.apache.flink.runtime.util.TestByteStreamStateHandleDeepCompare
;
import
org.apache.flink.util.InstantiationUtil
;
...
...
@@ -170,7 +171,8 @@ public class JobManagerHARecoveryTest extends TestLogger {
try
{
Scheduler
scheduler
=
new
Scheduler
(
TestingUtils
.
defaultExecutionContext
());
MySubmittedJobGraphStore
mySubmittedJobGraphStore
=
new
MySubmittedJobGraphStore
();
InMemorySubmittedJobGraphStore
submittedJobGraphStore
=
new
InMemorySubmittedJobGraphStore
();
submittedJobGraphStore
.
start
(
null
);
CompletedCheckpointStore
checkpointStore
=
new
RecoverableCompletedCheckpointStore
();
CheckpointIDCounter
checkpointCounter
=
new
StandaloneCheckpointIDCounter
();
CheckpointRecoveryFactory
checkpointStateFactory
=
new
MyCheckpointRecoveryFactory
(
checkpointStore
,
checkpointCounter
);
...
...
@@ -204,7 +206,7 @@ public class JobManagerHARecoveryTest extends TestLogger {
new
FixedDelayRestartStrategy
.
FixedDelayRestartStrategyFactory
(
Int
.
MaxValue
(),
100
),
timeout
,
myLeaderElectionService
,
myS
ubmittedJobGraphStore
,
s
ubmittedJobGraphStore
,
checkpointStateFactory
,
jobRecoveryTimeout
,
UnregisteredMetricGroups
.
createUnregisteredJobManagerMetricGroup
(),
...
...
@@ -286,7 +288,7 @@ public class JobManagerHARecoveryTest extends TestLogger {
// check that the job gets removed from the JobManager
Await
.
ready
(
jobRemoved
,
deadline
.
timeLeft
());
// but stays in the submitted job graph store
assertTrue
(
myS
ubmittedJobGraphStore
.
contains
(
jobGraph
.
getJobID
()));
assertTrue
(
s
ubmittedJobGraphStore
.
contains
(
jobGraph
.
getJobID
()));
Future
<
Object
>
jobRunning
=
gateway
.
ask
(
new
TestingJobManagerMessages
.
NotifyWhenJobStatus
(
jobGraph
.
getJobID
(),
JobStatus
.
RUNNING
),
deadline
.
timeLeft
());
...
...
@@ -306,7 +308,7 @@ public class JobManagerHARecoveryTest extends TestLogger {
Await
.
ready
(
jobFinished
,
deadline
.
timeLeft
());
// check that the job has been removed from the submitted job graph store
assertFalse
(
myS
ubmittedJobGraphStore
.
contains
(
jobGraph
.
getJobID
()));
assertFalse
(
s
ubmittedJobGraphStore
.
contains
(
jobGraph
.
getJobID
()));
// Check that state has been recovered
long
[]
recoveredStates
=
BlockingStatefulInvokable
.
getRecoveredStates
();
...
...
@@ -482,49 +484,6 @@ public class JobManagerHARecoveryTest extends TestLogger {
}
}
static
class
MySubmittedJobGraphStore
implements
SubmittedJobGraphStore
{
Map
<
JobID
,
SubmittedJobGraph
>
storedJobs
=
new
HashMap
<>();
@Override
public
void
start
(
SubmittedJobGraphListener
jobGraphListener
)
throws
Exception
{
}
@Override
public
void
stop
()
throws
Exception
{
}
@Override
public
SubmittedJobGraph
recoverJobGraph
(
JobID
jobId
)
throws
Exception
{
if
(
storedJobs
.
containsKey
(
jobId
))
{
return
storedJobs
.
get
(
jobId
);
}
else
{
return
null
;
}
}
@Override
public
void
putJobGraph
(
SubmittedJobGraph
jobGraph
)
throws
Exception
{
storedJobs
.
put
(
jobGraph
.
getJobId
(),
jobGraph
);
}
@Override
public
void
removeJobGraph
(
JobID
jobId
)
throws
Exception
{
storedJobs
.
remove
(
jobId
);
}
@Override
public
Collection
<
JobID
>
getJobIds
()
throws
Exception
{
return
storedJobs
.
keySet
();
}
boolean
contains
(
JobID
jobId
)
{
return
storedJobs
.
containsKey
(
jobId
);
}
}
public
static
class
BlockingInvokable
extends
AbstractInvokable
{
private
static
boolean
blocking
=
true
;
...
...
flink-runtime/src/test/java/org/apache/flink/runtime/testutils/InMemorySubmittedJobGraphStore.java
0 → 100644
浏览文件 @
87749ca4
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org.apache.flink.runtime.testutils
;
import
org.apache.flink.api.common.JobID
;
import
org.apache.flink.runtime.jobmanager.SubmittedJobGraph
;
import
org.apache.flink.runtime.jobmanager.SubmittedJobGraphStore
;
import
org.apache.flink.util.Preconditions
;
import
javax.annotation.Nullable
;
import
java.util.Collection
;
import
java.util.HashMap
;
import
java.util.Map
;
/**
* In-Memory implementation of {@link SubmittedJobGraphStore} for testing purposes.
*/
public
class
InMemorySubmittedJobGraphStore
implements
SubmittedJobGraphStore
{
private
final
Map
<
JobID
,
SubmittedJobGraph
>
storedJobs
=
new
HashMap
<>();
private
volatile
boolean
started
;
@Override
public
void
start
(
@Nullable
SubmittedJobGraphListener
jobGraphListener
)
throws
Exception
{
started
=
true
;
}
@Override
public
void
stop
()
throws
Exception
{
started
=
false
;
}
@Override
public
SubmittedJobGraph
recoverJobGraph
(
JobID
jobId
)
throws
Exception
{
verifyIsStarted
();
return
storedJobs
.
getOrDefault
(
jobId
,
null
);
}
@Override
public
void
putJobGraph
(
SubmittedJobGraph
jobGraph
)
throws
Exception
{
verifyIsStarted
();
storedJobs
.
put
(
jobGraph
.
getJobId
(),
jobGraph
);
}
@Override
public
void
removeJobGraph
(
JobID
jobId
)
throws
Exception
{
verifyIsStarted
();
storedJobs
.
remove
(
jobId
);
}
@Override
public
Collection
<
JobID
>
getJobIds
()
throws
Exception
{
verifyIsStarted
();
return
storedJobs
.
keySet
();
}
public
boolean
contains
(
JobID
jobId
)
{
verifyIsStarted
();
return
storedJobs
.
containsKey
(
jobId
);
}
private
void
verifyIsStarted
()
{
Preconditions
.
checkState
(
started
,
"Not running. Forgot to call start()?"
);
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录