Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
e217a8c6
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
e217a8c6
编写于
4月 16, 2020
作者:
I
ifndef-SleePy
提交者:
Piotr Nowojski
4月 17, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Revert "[hotfix] Harden ResumeCheckpointManuallyITCase"
This reverts commit
300263e1
.
上级
6adbe94b
变更
1
显示空白变更内容
内联
并排
Showing
1 changed file
with
12 addition
and
16 deletion
+12
-16
flink-tests/src/test/java/org/apache/flink/test/checkpointing/ResumeCheckpointManuallyITCase.java
...nk/test/checkpointing/ResumeCheckpointManuallyITCase.java
+12
-16
未找到文件。
flink-tests/src/test/java/org/apache/flink/test/checkpointing/ResumeCheckpointManuallyITCase.java
浏览文件 @
e217a8c6
...
@@ -29,7 +29,6 @@ import org.apache.flink.configuration.HighAvailabilityOptions;
...
@@ -29,7 +29,6 @@ import org.apache.flink.configuration.HighAvailabilityOptions;
import
org.apache.flink.contrib.streaming.state.RocksDBStateBackend
;
import
org.apache.flink.contrib.streaming.state.RocksDBStateBackend
;
import
org.apache.flink.runtime.jobgraph.JobGraph
;
import
org.apache.flink.runtime.jobgraph.JobGraph
;
import
org.apache.flink.runtime.jobgraph.SavepointRestoreSettings
;
import
org.apache.flink.runtime.jobgraph.SavepointRestoreSettings
;
import
org.apache.flink.runtime.state.CheckpointListener
;
import
org.apache.flink.runtime.state.StateBackend
;
import
org.apache.flink.runtime.state.StateBackend
;
import
org.apache.flink.runtime.state.filesystem.FsStateBackend
;
import
org.apache.flink.runtime.state.filesystem.FsStateBackend
;
import
org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration
;
import
org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration
;
...
@@ -298,15 +297,13 @@ public class ResumeCheckpointManuallyITCase extends TestLogger {
...
@@ -298,15 +297,13 @@ public class ResumeCheckpointManuallyITCase extends TestLogger {
private
static
String
runJobAndGetExternalizedCheckpoint
(
StateBackend
backend
,
File
checkpointDir
,
@Nullable
String
externalCheckpoint
,
ClusterClient
<?>
client
)
throws
Exception
{
private
static
String
runJobAndGetExternalizedCheckpoint
(
StateBackend
backend
,
File
checkpointDir
,
@Nullable
String
externalCheckpoint
,
ClusterClient
<?>
client
)
throws
Exception
{
JobGraph
initialJobGraph
=
getJobGraph
(
backend
,
externalCheckpoint
);
JobGraph
initialJobGraph
=
getJobGraph
(
backend
,
externalCheckpoint
);
NotifyingInfiniteTupleSource
.
countDownLatch
=
new
CountDownLatch
(
PARALLELISM
);
NotifyingInfiniteTupleSource
.
countDownLatch
=
new
CountDownLatch
(
PARALLELISM
);
NotifyingInfiniteTupleSource
.
checkpointCompletedLatch
=
new
CountDownLatch
(
PARALLELISM
);
ClientUtils
.
submitJob
(
client
,
initialJobGraph
);
ClientUtils
.
submitJob
(
client
,
initialJobGraph
);
// wait until all sources have been started
// wait until all sources have been started
NotifyingInfiniteTupleSource
.
countDownLatch
.
await
();
NotifyingInfiniteTupleSource
.
countDownLatch
.
await
();
// wait the checkpoint completing
NotifyingInfiniteTupleSource
.
checkpointCompletedLatch
.
await
();
waitUntilExternalizedCheckpointCreated
(
checkpointDir
,
initialJobGraph
.
getJobID
());
client
.
cancel
(
initialJobGraph
.
getJobID
()).
get
();
client
.
cancel
(
initialJobGraph
.
getJobID
()).
get
();
waitUntilCanceled
(
initialJobGraph
.
getJobID
(),
client
);
waitUntilCanceled
(
initialJobGraph
.
getJobID
(),
client
);
...
@@ -322,6 +319,16 @@ public class ResumeCheckpointManuallyITCase extends TestLogger {
...
@@ -322,6 +319,16 @@ public class ResumeCheckpointManuallyITCase extends TestLogger {
}
}
}
}
private
static
void
waitUntilExternalizedCheckpointCreated
(
File
checkpointDir
,
JobID
jobId
)
throws
InterruptedException
,
IOException
{
while
(
true
)
{
Thread
.
sleep
(
50
);
Optional
<
Path
>
externalizedCheckpoint
=
findExternalizedCheckpoint
(
checkpointDir
,
jobId
);
if
(
externalizedCheckpoint
.
isPresent
())
{
break
;
}
}
}
private
static
Optional
<
Path
>
findExternalizedCheckpoint
(
File
checkpointDir
,
JobID
jobId
)
throws
IOException
{
private
static
Optional
<
Path
>
findExternalizedCheckpoint
(
File
checkpointDir
,
JobID
jobId
)
throws
IOException
{
try
(
Stream
<
Path
>
checkpoints
=
Files
.
list
(
checkpointDir
.
toPath
().
resolve
(
jobId
.
toString
())))
{
try
(
Stream
<
Path
>
checkpoints
=
Files
.
list
(
checkpointDir
.
toPath
().
resolve
(
jobId
.
toString
())))
{
return
checkpoints
return
checkpoints
...
@@ -373,16 +380,12 @@ public class ResumeCheckpointManuallyITCase extends TestLogger {
...
@@ -373,16 +380,12 @@ public class ResumeCheckpointManuallyITCase extends TestLogger {
/**
/**
* Infinite source which notifies when all of its sub tasks have been started via the count down latch.
* Infinite source which notifies when all of its sub tasks have been started via the count down latch.
*/
*/
public
static
class
NotifyingInfiniteTupleSource
public
static
class
NotifyingInfiniteTupleSource
extends
ManualWindowSpeedITCase
.
InfiniteTupleSource
{
extends
ManualWindowSpeedITCase
.
InfiniteTupleSource
implements
CheckpointListener
{
private
static
final
long
serialVersionUID
=
8120981235081181746L
;
private
static
final
long
serialVersionUID
=
8120981235081181746L
;
private
static
CountDownLatch
countDownLatch
;
private
static
CountDownLatch
countDownLatch
;
private
static
CountDownLatch
checkpointCompletedLatch
;
public
NotifyingInfiniteTupleSource
(
int
numKeys
)
{
public
NotifyingInfiniteTupleSource
(
int
numKeys
)
{
super
(
numKeys
);
super
(
numKeys
);
}
}
...
@@ -395,12 +398,5 @@ public class ResumeCheckpointManuallyITCase extends TestLogger {
...
@@ -395,12 +398,5 @@ public class ResumeCheckpointManuallyITCase extends TestLogger {
super
.
run
(
out
);
super
.
run
(
out
);
}
}
@Override
public
void
notifyCheckpointComplete
(
long
checkpointId
)
throws
Exception
{
if
(
checkpointCompletedLatch
!=
null
)
{
checkpointCompletedLatch
.
countDown
();
}
}
}
}
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录