Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
LinuxSuRen
jenkins
提交
b5b37799
J
jenkins
项目概览
LinuxSuRen
/
jenkins
与 Fork 源项目一致
从无法访问的项目Fork
通知
2
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
J
jenkins
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
b5b37799
编写于
8月 26, 2015
作者:
J
Jesse Glick
浏览文件
操作
浏览文件
下载
差异文件
[JENKINS-10629] Merging #1670.
上级
79bf40bc
8e97fd1c
变更
7
隐藏空白更改
内联
并排
Showing
7 changed file
with
69 addition
and
64 deletion
+69
-64
changelog.html
changelog.html
+4
-1
core/pom.xml
core/pom.xml
+5
-0
core/src/main/java/hudson/FilePath.java
core/src/main/java/hudson/FilePath.java
+9
-23
core/src/main/java/hudson/org/apache/tools/tar/TarInputStream.java
...main/java/hudson/org/apache/tools/tar/TarInputStream.java
+2
-1
core/src/main/java/hudson/org/apache/tools/tar/TarOutputStream.java
...ain/java/hudson/org/apache/tools/tar/TarOutputStream.java
+3
-0
core/src/main/java/hudson/util/io/TarArchiver.java
core/src/main/java/hudson/util/io/TarArchiver.java
+14
-38
core/src/test/java/hudson/FilePathTest.java
core/src/test/java/hudson/FilePathTest.java
+32
-1
未找到文件。
changelog.html
浏览文件 @
b5b37799
...
...
@@ -58,6 +58,9 @@ Upcoming changes</a>
<li
class=
"major bug"
>
Race condition in triggers could cause various
<code>
NullPointerException
</code>
s.
(
<a
href=
"https://issues.jenkins-ci.org/browse/JENKINS-29790"
>
issue 29790
</a>
)
<li
class=
bug
>
Archiving of large artifacts. Tar implementation cannot handle files having a size >8GB.
(
<a
href=
"https://issues.jenkins-ci.org/browse/JENKINS-10629"
>
issue 10629
</a>
)
<li
class=
"rfe"
>
Allow plugins to augment or replace the plugin manager UI.
(
<a
href=
"https://github.com/jenkinsci/jenkins/pull/1788"
>
PR 1788
</a>
)
...
...
@@ -292,7 +295,7 @@ Upcoming changes</a>
Since 1.598 overrides of
<code>
Descriptor.getId
</code>
were not correctly handled by form binding, breaking at least the CloudBees Templates plugin.
(
<a
href=
"https://issues.jenkins-ci.org/browse/JENKINS-26781"
>
issue 26781
</a>
)
<li
class=
bug
>
<b>
Reverted in 1.611
</b>
. Archiving of large artifacts. Tar implementation cannot handle files having a size >8GB.
<b>
Reverted in 1.611
, reimplemented in 1.627
</b>
. Archiving of large artifacts. Tar implementation cannot handle files having a size >8GB.
(
<a
href=
"https://issues.jenkins-ci.org/browse/JENKINS-10629"
>
issue 10629
</a>
)
<li
class=
bug
>
The queue state was not updated between scheduling builds.
...
...
core/pom.xml
浏览文件 @
b5b37799
...
...
@@ -276,6 +276,11 @@ THE SOFTWARE.
<artifactId>
commons-beanutils
</artifactId>
<version>
1.8.3
</version>
</dependency>
<dependency>
<groupId>
org.apache.commons
</groupId>
<artifactId>
commons-compress
</artifactId>
<version>
1.10
</version>
</dependency>
<dependency>
<groupId>
javax.mail
</groupId>
<artifactId>
mail
</artifactId>
...
...
core/src/main/java/hudson/FilePath.java
浏览文件 @
b5b37799
...
...
@@ -33,7 +33,6 @@ import hudson.model.AbstractProject;
import
hudson.model.Computer
;
import
hudson.model.Item
;
import
hudson.model.TaskListener
;
import
hudson.org.apache.tools.tar.TarInputStream
;
import
hudson.os.PosixAPI
;
import
hudson.os.PosixException
;
import
hudson.remoting.Callable
;
...
...
@@ -70,7 +69,6 @@ import org.apache.commons.io.input.CountingInputStream;
import
org.apache.tools.ant.DirectoryScanner
;
import
org.apache.tools.ant.Project
;
import
org.apache.tools.ant.types.FileSet
;
import
org.apache.tools.tar.TarEntry
;
import
org.apache.tools.zip.ZipEntry
;
import
org.apache.tools.zip.ZipFile
;
import
org.kohsuke.stapler.Stapler
;
...
...
@@ -120,6 +118,8 @@ import static hudson.Util.*;
import
javax.annotation.Nonnull
;
import
javax.annotation.Nullable
;
import
jenkins.security.MasterToSlaveCallable
;
import
org.apache.commons.compress.archivers.tar.TarArchiveEntry
;
import
org.apache.commons.compress.archivers.tar.TarArchiveInputStream
;
import
org.jenkinsci.remoting.RoleChecker
;
import
org.jenkinsci.remoting.RoleSensitive
;
...
...
@@ -2268,12 +2268,15 @@ public final class FilePath implements Serializable {
/**
* Reads from a tar stream and stores obtained files to the base dir.
* @since TODO supports large files > 10 GB, migration to commons-compress
*/
private
void
readFromTar
(
String
name
,
File
baseDir
,
InputStream
in
)
throws
IOException
{
TarInputStream
t
=
new
TarInputStream
(
in
);
TarArchiveInputStream
t
=
new
TarArchiveInputStream
(
in
);
// TarInputStream t = new TarInputStream(in);
try
{
TarEntry
te
;
while
((
te
=
t
.
getNextEntry
())
!=
null
)
{
Tar
Archive
Entry
te
;
while
((
te
=
t
.
getNext
Tar
Entry
())
!=
null
)
{
File
f
=
new
File
(
baseDir
,
te
.
getName
());
if
(
te
.
isDirectory
())
{
mkdirs
(
f
);
...
...
@@ -2282,8 +2285,7 @@ public final class FilePath implements Serializable {
if
(
parent
!=
null
)
mkdirs
(
parent
);
writing
(
f
);
byte
linkFlag
=
(
Byte
)
LINKFLAG_FIELD
.
get
(
te
);
if
(
linkFlag
==
TarEntry
.
LF_SYMLINK
)
{
if
(
te
.
isSymbolicLink
())
{
new
FilePath
(
f
).
symlinkTo
(
te
.
getLinkName
(),
TaskListener
.
NULL
);
}
else
{
IOUtils
.
copy
(
t
,
f
);
...
...
@@ -2300,8 +2302,6 @@ public final class FilePath implements Serializable {
}
catch
(
InterruptedException
e
)
{
Thread
.
currentThread
().
interrupt
();
// process this later
throw
new
IOException
(
"Failed to extract "
+
name
,
e
);
}
catch
(
IllegalAccessException
e
)
{
throw
new
IOException
(
"Failed to extract "
+
name
,
e
);
}
finally
{
t
.
close
();
}
...
...
@@ -2725,20 +2725,6 @@ public final class FilePath implements Serializable {
}
};
private
static
final
Field
LINKFLAG_FIELD
=
getTarEntryLinkFlagField
();
private
static
Field
getTarEntryLinkFlagField
()
{
try
{
Field
f
=
TarEntry
.
class
.
getDeclaredField
(
"linkFlag"
);
f
.
setAccessible
(
true
);
return
f
;
}
catch
(
SecurityException
e
)
{
throw
new
AssertionError
(
e
);
}
catch
(
NoSuchFieldException
e
)
{
throw
new
AssertionError
(
e
);
}
}
/**
* Gets the {@link FilePath} representation of the "~" directory
* (User's home directory in the Unix sense) of the given channel.
...
...
core/src/main/java/hudson/org/apache/tools/tar/TarInputStream.java
浏览文件 @
b5b37799
...
...
@@ -37,8 +37,9 @@ import java.io.ByteArrayOutputStream;
* methods are provided to position at each successive entry in
* the archive, and the read each entry as a normal input stream
* using read().
*
*
@deprecated Use {@link org.apache.commons.compress.archivers.tar.TarArchiveInputStream} instead
*/
@Deprecated
public
class
TarInputStream
extends
FilterInputStream
{
// CheckStyle:VisibilityModifier OFF - bc
...
...
core/src/main/java/hudson/org/apache/tools/tar/TarOutputStream.java
浏览文件 @
b5b37799
...
...
@@ -35,8 +35,11 @@ import java.io.IOException;
* The TarOutputStream writes a UNIX tar archive as an OutputStream.
* Methods are provided to put entries, and then write their contents
* by writing to this stream using write().
*
* @deprecated Use {@link org.apache.commons.compress.archivers.tar.TarArchiveOutputStream} instead
*
*/
@Deprecated
public
class
TarOutputStream
extends
FilterOutputStream
{
/** Fail if a long file name is required in the archive. */
public
static
final
int
LONGFILE_ERROR
=
0
;
...
...
core/src/main/java/hudson/util/io/TarArchiver.java
浏览文件 @
b5b37799
...
...
@@ -37,6 +37,8 @@ import java.io.FileInputStream;
import
java.io.IOException
;
import
java.io.OutputStream
;
import
java.lang.reflect.Field
;
import
org.apache.commons.compress.archivers.tar.TarArchiveEntry
;
import
org.apache.commons.compress.archivers.tar.TarArchiveOutputStream
;
import
static
org
.
apache
.
tools
.
tar
.
TarConstants
.
LF_SYMLINK
;
...
...
@@ -47,24 +49,17 @@ import static org.apache.tools.tar.TarConstants.LF_SYMLINK;
*/
final
class
TarArchiver
extends
Archiver
{
private
final
byte
[]
buf
=
new
byte
[
8192
];
private
final
TarOutputStream
tar
;
private
final
Tar
Archive
OutputStream
tar
;
TarArchiver
(
OutputStream
out
)
{
tar
=
new
TarOutputStream
(
new
BufferedOutputStream
(
out
)
{
// TarOutputStream uses TarBuffer internally,
// which flushes the stream for each block. this creates unnecessary
// data stream fragmentation, and flush request to a remote, which slows things down.
@Override
public
void
flush
()
throws
IOException
{
// so don't do anything in flush
}
});
tar
.
setLongFileMode
(
TarOutputStream
.
LONGFILE_GNU
);
tar
=
new
TarArchiveOutputStream
(
out
);
tar
.
setBigNumberMode
(
TarArchiveOutputStream
.
BIGNUMBER_STAR
);
tar
.
setLongFileMode
(
TarArchiveOutputStream
.
LONGFILE_GNU
);
}
@Override
public
void
visitSymlink
(
File
link
,
String
target
,
String
relativePath
)
throws
IOException
{
Tar
Entry
e
=
new
Tar
Entry
(
relativePath
,
LF_SYMLINK
);
Tar
ArchiveEntry
e
=
new
TarArchive
Entry
(
relativePath
,
LF_SYMLINK
);
try
{
int
mode
=
IOUtils
.
mode
(
link
);
if
(
mode
!=
-
1
)
{
...
...
@@ -73,16 +68,11 @@ final class TarArchiver extends Archiver {
}
catch
(
PosixException
x
)
{
// ignore
}
e
.
setLinkName
(
target
);
try
{
StringBuffer
linkName
=
(
StringBuffer
)
LINKNAME_FIELD
.
get
(
e
);
linkName
.
setLength
(
0
);
linkName
.
append
(
target
);
}
catch
(
IllegalAccessException
x
)
{
throw
new
IOException
(
"Failed to set linkName"
,
x
);
}
tar
.
putNextEntry
(
e
);
tar
.
putArchiveEntry
(
e
);
tar
.
closeArchiveEntry
();
entriesWritten
++;
}
...
...
@@ -97,14 +87,14 @@ final class TarArchiver extends Archiver {
if
(
file
.
isDirectory
())
relativePath
+=
'/'
;
Tar
Entry
te
=
new
Tar
Entry
(
relativePath
);
Tar
ArchiveEntry
te
=
new
TarArchive
Entry
(
relativePath
);
int
mode
=
IOUtils
.
mode
(
file
);
if
(
mode
!=-
1
)
te
.
setMode
(
mode
);
te
.
setModTime
(
file
.
lastModified
());
if
(!
file
.
isDirectory
())
te
.
setSize
(
file
.
length
());
tar
.
put
Next
Entry
(
te
);
tar
.
put
Archive
Entry
(
te
);
if
(!
file
.
isDirectory
())
{
FileInputStream
in
=
new
FileInputStream
(
file
);
...
...
@@ -117,25 +107,11 @@ final class TarArchiver extends Archiver {
}
}
tar
.
closeEntry
();
tar
.
close
Archive
Entry
();
entriesWritten
++;
}
public
void
close
()
throws
IOException
{
tar
.
close
();
}
private
static
final
Field
LINKNAME_FIELD
=
getTarEntryLinkNameField
();
private
static
Field
getTarEntryLinkNameField
()
{
try
{
Field
f
=
TarEntry
.
class
.
getDeclaredField
(
"linkName"
);
f
.
setAccessible
(
true
);
return
f
;
}
catch
(
SecurityException
e
)
{
throw
new
AssertionError
(
e
);
}
catch
(
NoSuchFieldException
e
)
{
throw
new
AssertionError
(
e
);
}
}
}
core/src/test/java/hudson/FilePathTest.java
浏览文件 @
b5b37799
...
...
@@ -371,7 +371,7 @@ public class FilePathTest {
// Decompress
FilePath
outDir
=
new
FilePath
(
temp
.
newFolder
(
filePrefix
+
"_out"
));
final
FilePath
outFile
=
outDir
.
child
(
tempFile
.
getName
());
tmpDirPath
.
child
(
filePrefix
+
".tar"
).
untar
(
outDir
,
TarCompression
.
NONE
);
tmpDirPath
.
child
(
tarFile
.
getName
()
).
untar
(
outDir
,
TarCompression
.
NONE
);
assertEquals
(
"Result file after the roundtrip differs from the initial file"
,
new
FilePath
(
tempFile
).
digest
(),
outFile
.
digest
());
}
...
...
@@ -659,4 +659,35 @@ public class FilePathTest {
// test conflict subdir
src
.
moveAllChildrenTo
(
dst
);
}
@Issue
(
"JENKINS-10629"
)
@Test
public
void
testEOFbrokenFlush
()
throws
IOException
,
InterruptedException
{
final
File
srcFolder
=
temp
.
newFolder
(
"src"
);
// simulate magic structure with magic sizes:
// |- dir/pom.xml (2049)
// |- pom.xml (2049)
// \- small.tar (1537)
final
File
smallTar
=
new
File
(
srcFolder
,
"small.tar"
);
givenSomeContentInFile
(
smallTar
,
1537
);
final
File
dir
=
new
File
(
srcFolder
,
"dir"
);
dir
.
mkdirs
();
final
File
pomFile
=
new
File
(
dir
,
"pom.xml"
);
givenSomeContentInFile
(
pomFile
,
2049
);
FileUtils
.
copyFileToDirectory
(
pomFile
,
srcFolder
);
final
File
archive
=
temp
.
newFile
(
"archive.tar"
);
// Compress archive
final
FilePath
tmpDirPath
=
new
FilePath
(
srcFolder
);
int
tarred
=
tmpDirPath
.
tar
(
new
FileOutputStream
(
archive
),
"**"
);
assertEquals
(
"One file should have been compressed"
,
3
,
tarred
);
// Decompress
final
File
dstFolder
=
temp
.
newFolder
(
"dst"
);
dstFolder
.
mkdirs
();
FilePath
outDir
=
new
FilePath
(
dstFolder
);
// and now fail when flush is bad!
tmpDirPath
.
child
(
"../"
+
archive
.
getName
()).
untar
(
outDir
,
TarCompression
.
NONE
);
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录