Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
614abd29
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
614abd29
编写于
3月 10, 2017
作者:
金
金竹
提交者:
Stephan Ewen
3月 20, 2017
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[FLINK-5995] [checkpoints] Fix serializer initialization for Operator State
This closes #3503
上级
486f7249
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
36 addition
and
3 deletion
+36
-3
flink-runtime/src/main/java/org/apache/flink/runtime/state/AbstractStateBackend.java
.../org/apache/flink/runtime/state/AbstractStateBackend.java
+1
-1
flink-runtime/src/main/java/org/apache/flink/runtime/state/DefaultOperatorStateBackend.java
...ache/flink/runtime/state/DefaultOperatorStateBackend.java
+10
-2
flink-runtime/src/test/java/org/apache/flink/runtime/state/OperatorStateBackendTest.java
.../apache/flink/runtime/state/OperatorStateBackendTest.java
+25
-0
未找到文件。
flink-runtime/src/main/java/org/apache/flink/runtime/state/AbstractStateBackend.java
浏览文件 @
614abd29
...
...
@@ -95,7 +95,7 @@ public abstract class AbstractStateBackend implements StateBackend, java.io.Seri
Environment
env
,
String
operatorIdentifier
)
throws
Exception
{
return
new
DefaultOperatorStateBackend
(
env
.
getUserClassLoader
());
return
new
DefaultOperatorStateBackend
(
env
.
getUserClassLoader
()
,
env
.
getExecutionConfig
()
);
}
// ------------------------------------------------------------------------
...
...
flink-runtime/src/main/java/org/apache/flink/runtime/state/DefaultOperatorStateBackend.java
浏览文件 @
614abd29
...
...
@@ -23,6 +23,7 @@ import org.apache.flink.annotation.Internal;
import
org.apache.flink.api.common.state.ListState
;
import
org.apache.flink.api.common.state.ListStateDescriptor
;
import
org.apache.flink.api.common.typeutils.TypeSerializer
;
import
org.apache.flink.api.common.ExecutionConfig
;
import
org.apache.flink.core.fs.CloseableRegistry
;
import
org.apache.flink.core.fs.FSDataInputStream
;
import
org.apache.flink.core.fs.FSDataOutputStream
;
...
...
@@ -56,15 +57,20 @@ public class DefaultOperatorStateBackend implements OperatorStateBackend {
private
final
CloseableRegistry
closeStreamOnCancelRegistry
;
private
final
JavaSerializer
<
Serializable
>
javaSerializer
;
private
final
ClassLoader
userClassloader
;
private
final
ExecutionConfig
executionConfig
;
public
DefaultOperatorStateBackend
(
ClassLoader
userClassLoader
)
throws
IOException
{
public
DefaultOperatorStateBackend
(
ClassLoader
userClassLoader
,
ExecutionConfig
executionConfig
)
throws
IOException
{
this
.
closeStreamOnCancelRegistry
=
new
CloseableRegistry
();
this
.
userClassloader
=
Preconditions
.
checkNotNull
(
userClassLoader
);
this
.
executionConfig
=
executionConfig
;
this
.
javaSerializer
=
new
JavaSerializer
<>();
this
.
registeredStates
=
new
HashMap
<>();
}
public
ExecutionConfig
getExecutionConfig
()
{
return
executionConfig
;
}
@Override
public
Set
<
String
>
getRegisteredStateNames
()
{
return
registeredStates
.
keySet
();
...
...
@@ -106,6 +112,8 @@ public class DefaultOperatorStateBackend implements OperatorStateBackend {
Preconditions
.
checkNotNull
(
stateDescriptor
);
stateDescriptor
.
initializeSerializerUnlessSet
(
getExecutionConfig
());
String
name
=
Preconditions
.
checkNotNull
(
stateDescriptor
.
getName
());
TypeSerializer
<
S
>
partitionStateSerializer
=
Preconditions
.
checkNotNull
(
stateDescriptor
.
getElementSerializer
());
...
...
flink-runtime/src/test/java/org/apache/flink/runtime/state/OperatorStateBackendTest.java
浏览文件 @
614abd29
...
...
@@ -17,6 +17,7 @@
package
org.apache.flink.runtime.state
;
import
org.apache.flink.api.common.ExecutionConfig
;
import
org.apache.flink.api.common.JobID
;
import
org.apache.flink.api.common.state.ListState
;
import
org.apache.flink.api.common.state.ListStateDescriptor
;
...
...
@@ -28,6 +29,7 @@ import org.junit.Assert;
import
org.junit.Test
;
import
java.io.Serializable
;
import
java.io.File
;
import
java.util.Collections
;
import
java.util.Iterator
;
import
java.util.concurrent.RunnableFuture
;
...
...
@@ -45,6 +47,8 @@ public class OperatorStateBackendTest {
static
Environment
createMockEnvironment
()
{
Environment
env
=
mock
(
Environment
.
class
);
ExecutionConfig
config
=
mock
(
ExecutionConfig
.
class
);
when
(
env
.
getExecutionConfig
()).
thenReturn
(
config
);
when
(
env
.
getUserClassLoader
()).
thenReturn
(
Thread
.
currentThread
().
getContextClassLoader
());
return
env
;
}
...
...
@@ -63,6 +67,27 @@ public class OperatorStateBackendTest {
assertTrue
(
operatorStateBackend
.
getRegisteredStateNames
().
isEmpty
());
}
@Test
public
void
testRegisterStatesWithoutTypeSerializer
()
throws
Exception
{
DefaultOperatorStateBackend
operatorStateBackend
=
createNewOperatorStateBackend
();
ListStateDescriptor
<
File
>
stateDescriptor
=
new
ListStateDescriptor
<>(
"test"
,
File
.
class
);
ListStateDescriptor
<
String
>
stateDescriptor2
=
new
ListStateDescriptor
<>(
"test2"
,
String
.
class
);
ListState
<
File
>
listState
=
operatorStateBackend
.
getOperatorState
(
stateDescriptor
);
assertNotNull
(
listState
);
ListState
<
String
>
listState2
=
operatorStateBackend
.
getOperatorState
(
stateDescriptor2
);
assertNotNull
(
listState2
);
assertEquals
(
2
,
operatorStateBackend
.
getRegisteredStateNames
().
size
());
Iterator
<
String
>
it
=
listState2
.
get
().
iterator
();
assertTrue
(!
it
.
hasNext
());
listState2
.
add
(
"kevin"
);
listState2
.
add
(
"sunny"
);
it
=
listState2
.
get
().
iterator
();
assertEquals
(
"kevin"
,
it
.
next
());
assertEquals
(
"sunny"
,
it
.
next
());
assertTrue
(!
it
.
hasNext
());
}
@Test
public
void
testRegisterStates
()
throws
Exception
{
DefaultOperatorStateBackend
operatorStateBackend
=
createNewOperatorStateBackend
();
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录