提交 8290004d 编写于 作者: S Shixiong Zhu

[SPARK-13693][STREAMING][TESTS] Stop StreamingContext before deleting checkpoint dir

## What changes were proposed in this pull request?

Stop StreamingContext before deleting checkpoint dir to avoid the race condition that deleting the checkpoint dir and writing checkpoint happen at the same time.

The flaky test log is here: https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7/256/testReport/junit/org.apache.spark.streaming/MapWithStateSuite/_It_is_not_a_test_/

## How was this patch tested?

unit tests

Author: Shixiong Zhu <shixiong@databricks.com>

Closes #11531 from zsxwing/SPARK-13693.
上级 adce5ee7
......@@ -43,10 +43,10 @@ class MapWithStateSuite extends SparkFunSuite
}
after {
StreamingContext.getActive().foreach { _.stop(stopSparkContext = false) }
if (checkpointDir != null) {
Utils.deleteRecursively(checkpointDir)
}
StreamingContext.getActive().foreach { _.stop(stopSparkContext = false) }
}
override def beforeAll(): Unit = {
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册