Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
flink
提交
cd899f3b
F
flink
项目概览
doujutun3207
/
flink
与 Fork 源项目一致
从无法访问的项目Fork
通知
24
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
F
flink
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
cd899f3b
编写于
11月 03, 2015
作者:
F
Fabian Hueske
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[FLINK-2972] [JavaAPI] Remove Chill dependency from flink-java.
This closes #1331
上级
d5a6b13a
变更
5
隐藏空白更改
内联
并排
Showing
5 changed file
with
46 addition
and
16 deletion
+46
-16
flink-core/pom.xml
flink-core/pom.xml
+0
-6
flink-java/pom.xml
flink-java/pom.xml
+10
-7
flink-java/src/main/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoSerializer.java
...flink/api/java/typeutils/runtime/kryo/KryoSerializer.java
+23
-2
flink-runtime/pom.xml
flink-runtime/pom.xml
+12
-0
flink-scala/pom.xml
flink-scala/pom.xml
+1
-1
未找到文件。
flink-core/pom.xml
浏览文件 @
cd899f3b
...
...
@@ -52,12 +52,6 @@ under the License.
<artifactId>
kryo
</artifactId>
</dependency>
<dependency>
<groupId>
com.twitter
</groupId>
<artifactId>
chill_${scala.binary.version}
</artifactId>
<version>
${chill.version}
</version>
</dependency>
<dependency>
<groupId>
com.google.guava
</groupId>
<artifactId>
guava
</artifactId>
...
...
flink-java/pom.xml
浏览文件 @
cd899f3b
...
...
@@ -65,13 +65,7 @@ under the License.
<dependency>
<groupId>
com.twitter
</groupId>
<artifactId>
chill_${scala.binary.version}
</artifactId>
<version>
${chill.version}
</version>
</dependency>
<dependency>
<groupId>
com.twitter
</groupId>
<artifactId>
chill-avro_${scala.binary.version}
</artifactId>
<artifactId>
chill-java
</artifactId>
<version>
${chill.version}
</version>
</dependency>
...
...
@@ -110,6 +104,15 @@ under the License.
<type>
test-jar
</type>
<scope>
test
</scope>
</dependency>
<dependency>
<groupId>
com.twitter
</groupId>
<artifactId>
chill_${scala.binary.version}
</artifactId>
<version>
${chill.version}
</version>
<!-- For execution, Chill is added to the classpath through flink-runtime. -->
<scope>
test
</scope>
</dependency>
</dependencies>
<!-- Because flink-scala and flink-avro uses it in tests -->
...
...
flink-java/src/main/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoSerializer.java
浏览文件 @
cd899f3b
...
...
@@ -26,7 +26,6 @@ import com.esotericsoftware.kryo.io.Input;
import
com.esotericsoftware.kryo.io.Output
;
import
com.esotericsoftware.kryo.serializers.JavaSerializer
;
import
com.google.common.base.Preconditions
;
import
com.twitter.chill.ScalaKryoInstantiator
;
import
org.apache.avro.generic.GenericData
;
import
org.apache.flink.api.common.ExecutionConfig
;
...
...
@@ -42,6 +41,8 @@ import java.io.ByteArrayInputStream;
import
java.io.ByteArrayOutputStream
;
import
java.io.EOFException
;
import
java.io.IOException
;
import
java.lang.reflect.InvocationTargetException
;
import
java.lang.reflect.Method
;
import
java.lang.reflect.Modifier
;
import
java.util.LinkedHashMap
;
import
java.util.LinkedHashSet
;
...
...
@@ -281,9 +282,29 @@ public class KryoSerializer<T> extends TypeSerializer<T> {
// --------------------------------------------------------------------------------------------
private
Kryo
getKryoInstance
()
{
try
{
// check if ScalaKryoInstantiator is in class path (coming from Twitter's Chill library).
// This will be true if Flink's Scala API is used.
Class
<?>
chillInstantiatorClazz
=
Class
.
forName
(
"com.twitter.chill.ScalaKryoInstantiator"
);
Object
chillInstantiator
=
chillInstantiatorClazz
.
newInstance
();
// obtain a Kryo instance through Twitter Chill
Method
m
=
chillInstantiatorClazz
.
getMethod
(
"newKryo"
);
return
(
Kryo
)
m
.
invoke
(
chillInstantiator
);
}
catch
(
ClassNotFoundException
|
InstantiationException
|
NoSuchMethodException
|
IllegalAccessException
|
InvocationTargetException
e
)
{
// Chill must be in the classpath. It is added as a dependency to flink-runtime.
throw
new
RuntimeException
(
"Could not instantiate Kryo instance from Chill."
,
e
);
}
}
private
void
checkKryoInitialized
()
{
if
(
this
.
kryo
==
null
)
{
this
.
kryo
=
new
ScalaKryoInstantiator
().
newKryo
();
this
.
kryo
=
getKryoInstance
();
// Throwable and all subclasses should be serialized via java serialization
kryo
.
addDefaultSerializer
(
Throwable
.
class
,
new
JavaSerializer
());
...
...
flink-runtime/pom.xml
浏览文件 @
cd899f3b
...
...
@@ -180,6 +180,18 @@ under the License.
<artifactId>
zookeeper
</artifactId>
</dependency>
<!--
The KryoSerializer dynamically loads Kryo instances via Chill and requires that Chill
is in the classpath. Because we do not want to have transitive Scala dependencies
in Flink's API modules (such as flink-java) due to Chill, Chill is added to flink-runtime
to ensure that Chill is always present in the classpath.
-->
<dependency>
<groupId>
com.twitter
</groupId>
<artifactId>
chill_${scala.binary.version}
</artifactId>
<version>
${chill.version}
</version>
</dependency>
<dependency>
<groupId>
org.apache.flink
</groupId>
<artifactId>
flink-shaded-curator-recipes
</artifactId>
...
...
flink-scala/pom.xml
浏览文件 @
cd899f3b
...
...
@@ -69,7 +69,7 @@ under the License.
<dependency>
<groupId>
org.ow2.asm
</groupId>
<artifactId>
asm
</artifactId>
<version>
${asm.version}
</version>
<version>
${asm.version}
</version>
</dependency>
<dependency>
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录