Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
doujutun3207
spark
提交
545165e8
S
spark
项目概览
doujutun3207
/
spark
与 Fork 源项目一致
从无法访问的项目Fork
通知
2
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
spark
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
545165e8
编写于
7月 30, 2012
作者:
M
Matei Zaharia
浏览文件
操作
浏览文件
下载
差异文件
Merge pull request #158 from JoshRosen/fix/assembly-akka-config
Merge Akka reference.conf files in sbt assembly task
上级
3ee2530c
509b721d
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
55 addition
and
2 deletion
+55
-2
project/SparkBuild.scala
project/SparkBuild.scala
+55
-2
未找到文件。
project/SparkBuild.scala
浏览文件 @
545165e8
import
sbt._
import
Keys._
import
classpath.ClasspathUtilities.isArchive
import
java.io.FileOutputStream
import
sbtassembly.Plugin._
import
AssemblyKeys._
...
...
@@ -69,12 +71,12 @@ object SparkBuild extends Build {
"cc.spray"
%
"spray-can"
%
"1.0-M2.1"
,
"cc.spray"
%
"spray-server"
%
"1.0-M2.1"
)
)
++
assemblySettings
++
extraAssemblySettings
)
++
assemblySettings
++
extraAssemblySettings
++
mergeSettings
def
replSettings
=
sharedSettings
++
Seq
(
name
:=
"spark-repl"
,
libraryDependencies
<+=
scalaVersion
(
"org.scala-lang"
%
"scala-compiler"
%
_
)
)
++
assemblySettings
++
extraAssemblySettings
)
++
assemblySettings
++
extraAssemblySettings
++
mergeSettings
def
examplesSettings
=
sharedSettings
++
Seq
(
name
:=
"spark-examples"
...
...
@@ -82,6 +84,57 @@ object SparkBuild extends Build {
def
bagelSettings
=
sharedSettings
++
Seq
(
name
:=
"spark-bagel"
)
// Fix for "No configuration setting found for key 'akka.version'" exception
// when running Spark from the jar generated by the "assembly" task; see
// http://letitcrash.com/post/21025950392/howto-sbt-assembly-vs-reference-conf
lazy
val
merge
=
TaskKey
[
File
](
"merge-reference"
,
"merge all reference.conf"
)
lazy
val
mergeSettings
:
Seq
[
Project.Setting
[
_
]]
=
Seq
(
merge
<<=
(
fullClasspath
in
assembly
)
map
{
c
=>
// collect from all elements of the full classpath
val
(
libs
,
dirs
)
=
c
map
(
_
.
data
)
partition
(
isArchive
)
// goal is to simply concatenate files here
val
dest
=
file
(
"reference.conf"
)
val
out
=
new
FileOutputStream
(
dest
)
val
append
=
IO
.
transfer
(
_:
File
,
out
)
try
{
// first collect from managed sources
(
dirs
*
"reference.conf"
).
get
foreach
append
// then from dependency jars by unzipping and
// collecting reference.conf if present
for
(
lib
<-
libs
)
{
IO
withTemporaryDirectory
{
dir
=>
IO
.
unzip
(
lib
,
dir
,
"reference.conf"
)
(
dir
*
"reference.conf"
).
get
foreach
append
}
}
// return merged file location as task result
dest
}
finally
{
out
.
close
()
}
},
// get rid of the individual files from jars
excludedFiles
in
assembly
<<=
(
excludedFiles
in
assembly
)
{
(
old
)
=>
(
bases
)
=>
old
(
bases
)
++
(
bases
flatMap
(
base
=>
(
base
/
"reference.conf"
).
get
))
},
// tell sbt-assembly to include our merged file
assembledMappings
in
assembly
<<=
(
assembledMappings
in
assembly
,
merge
)
map
{
(
old
,
merged
)
=>
(
f
)
=>
old
(
f
)
:+(
merged
,
"reference.conf"
)
}
)
def
extraAssemblySettings
()
=
Seq
(
test
in
assembly
:=
{})
++
Seq
(
mergeStrategy
in
assembly
:=
{
case
m
if
m
.
toLowerCase
.
endsWith
(
"manifest.mf"
)
=>
MergeStrategy
.
discard
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录