Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
5ffcfcaf
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
5ffcfcaf
编写于
12月 25, 2020
作者:
J
Jack
提交者:
GitHub
12月 25, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[FLINK-20710][hive] Hive reading fail when splits on different FileSystem nameservices
This closes #14452
上级
f5e829c4
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
2 addition
and
8 deletion
+2
-8
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/HiveSourceFileEnumerator.java
...pache/flink/connectors/hive/HiveSourceFileEnumerator.java
+1
-4
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/read/HiveTableInputFormat.java
...ache/flink/connectors/hive/read/HiveTableInputFormat.java
+1
-4
未找到文件。
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/HiveSourceFileEnumerator.java
浏览文件 @
5ffcfcaf
...
...
@@ -62,13 +62,10 @@ public class HiveSourceFileEnumerator implements FileEnumerator {
List
<
HiveTablePartition
>
partitions
,
JobConf
jobConf
)
throws
IOException
{
List
<
HiveSourceSplit
>
hiveSplits
=
new
ArrayList
<>();
FileSystem
fs
=
null
;
for
(
HiveTablePartition
partition
:
partitions
)
{
StorageDescriptor
sd
=
partition
.
getStorageDescriptor
();
org
.
apache
.
hadoop
.
fs
.
Path
inputPath
=
new
org
.
apache
.
hadoop
.
fs
.
Path
(
sd
.
getLocation
());
if
(
fs
==
null
)
{
fs
=
inputPath
.
getFileSystem
(
jobConf
);
}
FileSystem
fs
=
inputPath
.
getFileSystem
(
jobConf
);
// it's possible a partition exists in metastore but the data has been removed
if
(!
fs
.
exists
(
inputPath
))
{
continue
;
...
...
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/read/HiveTableInputFormat.java
浏览文件 @
5ffcfcaf
...
...
@@ -285,13 +285,10 @@ public class HiveTableInputFormat extends HadoopInputFormatCommonBase<RowData, H
JobConf
jobConf
)
throws
IOException
{
List
<
HiveTableInputSplit
>
hiveSplits
=
new
ArrayList
<>();
int
splitNum
=
0
;
FileSystem
fs
=
null
;
for
(
HiveTablePartition
partition
:
partitions
)
{
StorageDescriptor
sd
=
partition
.
getStorageDescriptor
();
Path
inputPath
=
new
Path
(
sd
.
getLocation
());
if
(
fs
==
null
)
{
fs
=
inputPath
.
getFileSystem
(
jobConf
);
}
FileSystem
fs
=
inputPath
.
getFileSystem
(
jobConf
);
// it's possible a partition exists in metastore but the data has been removed
if
(!
fs
.
exists
(
inputPath
))
{
continue
;
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录