Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
39562691
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
39562691
编写于
6月 27, 2017
作者:
Z
zhangminglei
提交者:
zentol
7月 01, 2017
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[FLINK-6655] Add validateAndNormalizeUri method to MemoryArchivist
This closes #4156.
上级
3f0ac26e
变更
1
显示空白变更内容
内联
并排
Showing
1 changed file
with
55 addition
and
11 deletion
+55
-11
flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/MemoryArchivist.scala
...org/apache/flink/runtime/jobmanager/MemoryArchivist.scala
+55
-11
未找到文件。
flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/MemoryArchivist.scala
浏览文件 @
39562691
...
...
@@ -18,13 +18,14 @@
package
org.apache.flink.runtime.jobmanager
import
java.io.IOException
import
java.net.URI
import
java.util
import
akka.actor.ActorRef
import
grizzled.slf4j.Logger
import
org.apache.flink.api.common.JobID
import
org.apache.flink.configuration.Configuration
import
org.apache.flink.core.fs.Path
import
org.apache.flink.core.fs.
{
FileSystem
,
Path
}
import
org.apache.flink.runtime.jobgraph.JobStatus
import
org.apache.flink.runtime.messages.accumulators._
import
org.apache.flink.runtime.webmonitor.WebMonitorUtils
...
...
@@ -34,7 +35,6 @@ import org.apache.flink.runtime.executiongraph.{ArchivedExecutionGraph, Executio
import
org.apache.flink.runtime.history.FsJobArchivist
import
org.apache.flink.runtime.messages.ArchiveMessages._
import
org.apache.flink.runtime.messages.JobManagerMessages._
import
org.apache.flink.runtime.state.filesystem.FsStateBackend
import
scala.collection.mutable
import
scala.concurrent.future
...
...
@@ -198,7 +198,7 @@ class MemoryArchivist(
// so we aren't archiving it yet.
if
(
archivePath
.
isDefined
&&
graph
.
getState
.
isGloballyTerminalState
)
{
try
{
val
p
=
FsStateBackend
.
validateAndNormalizeUri
(
archivePath
.
get
.
toUri
)
val
p
=
validateAndNormalizeUri
(
archivePath
.
get
.
toUri
)
future
{
try
{
FsJobArchivist
.
archiveJob
(
p
,
graph
)
...
...
@@ -255,4 +255,48 @@ class MemoryArchivist(
graphs
.
remove
(
jobID
)
}
}
/**
* Checks and normalizes the archive path URI. This method first checks the validity of the
* URI (scheme, path, availability of a matching file system) and then normalizes the URL
* to a path.
*
* If the URI does not include an authority, but the file system configured for the URI has an
* authority, then the normalized path will include this authority.
*
* @param archivePathUri The URI to check and normalize.
* @return a normalized URI as a Path.
*
* @throws IllegalArgumentException Thrown, if the URI misses schema or path.
* @throws IOException Thrown, if no file system can be found for the URI's scheme.
*/
@throws
[
IOException
]
private
def
validateAndNormalizeUri
(
archivePathUri
:
URI
)
:
Path
=
{
val
scheme
=
archivePathUri
.
getScheme
val
path
=
archivePathUri
.
getPath
// some validity checks
if
(
scheme
==
null
)
{
throw
new
IllegalArgumentException
(
"The scheme (hdfs://, file://, etc) is null. "
+
"Please specify the file system scheme explicitly in the URI: "
+
archivePathUri
)
}
if
(
path
==
null
)
{
throw
new
IllegalArgumentException
(
"The path to store the job archives is null. "
+
"Please specify a directory path for storing job archives. and the URI is: "
+
archivePathUri
)
}
if
(
path
.
length
==
0
||
path
==
"/"
)
{
throw
new
IllegalArgumentException
(
"Cannot use the root directory for storing job archives."
)
}
if
(!
FileSystem
.
isFlinkSupportedScheme
(
archivePathUri
.
getScheme
))
{
// skip verification checks for non-flink supported filesystem
// this is because the required filesystem classes may not be available to the flink client
throw
new
IllegalArgumentException
(
"No file system found with scheme "
+
scheme
+
", referenced in file URI '"
+
archivePathUri
.
toString
+
"'."
)
}
new
Path
(
archivePathUri
)
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录