Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
李少辉-开发者
gitlab-foss
提交
4419d7ea
G
gitlab-foss
项目概览
李少辉-开发者
/
gitlab-foss
通知
15
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
G
gitlab-foss
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
4419d7ea
编写于
3月 06, 2018
作者:
N
Nick Thomas
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Implement foreground verification of CI artifacts
上级
98c8c90e
变更
7
隐藏空白更改
内联
并排
Showing
7 changed file
with
119 addition
and
0 deletion
+119
-0
changelogs/unreleased/43949-verify-job-artifacts.yml
changelogs/unreleased/43949-verify-job-artifacts.yml
+5
-0
doc/administration/raketasks/check.md
doc/administration/raketasks/check.md
+4
-0
lib/gitlab/verify/job_artifacts.rb
lib/gitlab/verify/job_artifacts.rb
+27
-0
lib/tasks/gitlab/artifacts/check.rake
lib/tasks/gitlab/artifacts/check.rake
+8
-0
spec/factories/ci/job_artifacts.rb
spec/factories/ci/job_artifacts.rb
+6
-0
spec/lib/gitlab/verify/job_artifacts_spec.rb
spec/lib/gitlab/verify/job_artifacts_spec.rb
+35
-0
spec/tasks/gitlab/artifacts/check_rake_spec.rb
spec/tasks/gitlab/artifacts/check_rake_spec.rb
+34
-0
未找到文件。
changelogs/unreleased/43949-verify-job-artifacts.yml
0 → 100644
浏览文件 @
4419d7ea
---
title
:
Implement foreground verification of CI artifacts
merge_request
:
17578
author
:
type
:
added
doc/administration/raketasks/check.md
浏览文件 @
4419d7ea
...
...
@@ -84,12 +84,14 @@ checks using those checksums can be run. These checks also detect missing files.
Currently, integrity checks are supported for the following types of file:
*
CI artifacts
*
LFS objects
*
User uploads
**Omnibus Installation**
```
sudo gitlab-rake gitlab:artifacts:check
sudo gitlab-rake gitlab:lfs:check
sudo gitlab-rake gitlab:uploads:check
```
...
...
@@ -97,6 +99,7 @@ sudo gitlab-rake gitlab:uploads:check
**Source Installation**
```
bash
sudo
-u
git
-H
bundle
exec
rake gitlab:artifacts:check
RAILS_ENV
=
production
sudo
-u
git
-H
bundle
exec
rake gitlab:lfs:check
RAILS_ENV
=
production
sudo
-u
git
-H
bundle
exec
rake gitlab:uploads:check
RAILS_ENV
=
production
```
...
...
@@ -112,6 +115,7 @@ Variable | Type | Description
`VERBOSE`
| boolean | Causes failures to be listed individually, rather than being summarized.
```
bash
sudo
gitlab-rake gitlab:artifacts:check
BATCH
=
100
ID_FROM
=
50
ID_TO
=
250
sudo
gitlab-rake gitlab:lfs:check
BATCH
=
100
ID_FROM
=
50
ID_TO
=
250
sudo
gitlab-rake gitlab:uploads:check
BATCH
=
100
ID_FROM
=
50
ID_TO
=
250
```
...
...
lib/gitlab/verify/job_artifacts.rb
0 → 100644
浏览文件 @
4419d7ea
module
Gitlab
module
Verify
class
JobArtifacts
<
BatchVerifier
def
name
'Job artifacts'
end
def
describe
(
object
)
"Job artifact:
#{
object
.
id
}
"
end
private
def
relation
::
Ci
::
JobArtifact
.
all
end
def
expected_checksum
(
artifact
)
artifact
.
file_sha256
end
def
actual_checksum
(
artifact
)
Digest
::
SHA256
.
file
(
artifact
.
file
.
path
).
hexdigest
end
end
end
end
lib/tasks/gitlab/artifacts/check.rake
0 → 100644
浏览文件 @
4419d7ea
namespace
:gitlab
do
namespace
:artifacts
do
desc
'GitLab | Artifacts | Check integrity of uploaded job artifacts'
task
check: :environment
do
Gitlab
::
Verify
::
RakeTask
.
run!
(
Gitlab
::
Verify
::
JobArtifacts
)
end
end
end
spec/factories/ci/job_artifacts.rb
浏览文件 @
4419d7ea
...
...
@@ -35,5 +35,11 @@ FactoryBot.define do
Rails
.
root
.
join
(
'spec/fixtures/trace/sample_trace'
),
'text/plain'
)
end
end
trait
:correct_checksum
do
after
(
:build
)
do
|
artifact
,
evaluator
|
artifact
.
file_sha256
=
Digest
::
SHA256
.
file
(
artifact
.
file
.
path
).
hexdigest
end
end
end
end
spec/lib/gitlab/verify/job_artifacts_spec.rb
0 → 100644
浏览文件 @
4419d7ea
require
'spec_helper'
describe
Gitlab
::
Verify
::
JobArtifacts
do
include
GitlabVerifyHelpers
it_behaves_like
'Gitlab::Verify::BatchVerifier subclass'
do
let!
(
:objects
)
{
create_list
(
:ci_job_artifact
,
3
,
:archive
)
}
end
describe
'#run_batches'
do
let
(
:failures
)
{
collect_failures
}
let
(
:failure
)
{
failures
[
artifact
]
}
let!
(
:artifact
)
{
create
(
:ci_job_artifact
,
:archive
,
:correct_checksum
)
}
it
'passes artifacts with the correct file'
do
expect
(
failures
).
to
eq
({})
end
it
'fails artifacts with a missing file'
do
FileUtils
.
rm_f
(
artifact
.
file
.
path
)
expect
(
failures
.
keys
).
to
contain_exactly
(
artifact
)
expect
(
failure
).
to
be_a
(
Errno
::
ENOENT
)
expect
(
failure
.
to_s
).
to
include
(
artifact
.
file
.
path
)
end
it
'fails artifacts with a mismatched checksum'
do
File
.
truncate
(
artifact
.
file
.
path
,
0
)
expect
(
failures
.
keys
).
to
contain_exactly
(
artifact
)
expect
(
failure
.
to_s
).
to
include
(
'Checksum mismatch'
)
end
end
end
spec/tasks/gitlab/artifacts/check_rake_spec.rb
0 → 100644
浏览文件 @
4419d7ea
require
'rake_helper'
describe
'gitlab:artifacts rake tasks'
do
describe
'check'
do
let!
(
:artifact
)
{
create
(
:ci_job_artifact
,
:archive
,
:correct_checksum
)
}
before
do
Rake
.
application
.
rake_require
(
'tasks/gitlab/artifacts/check'
)
stub_env
(
'VERBOSE'
=>
'true'
)
end
it
'outputs the integrity check for each batch'
do
expect
{
run_rake_task
(
'gitlab:artifacts:check'
)
}.
to
output
(
/Failures: 0/
).
to_stdout
end
it
'errors out about missing files on the file system'
do
FileUtils
.
rm_f
(
artifact
.
file
.
path
)
expect
{
run_rake_task
(
'gitlab:artifacts:check'
)
}.
to
output
(
/No such file.*
#{
Regexp
.
quote
(
artifact
.
file
.
path
)
}
/
).
to_stdout
end
it
'errors out about invalid checksum'
do
artifact
.
update_column
(
:file_sha256
,
'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
)
expect
{
run_rake_task
(
'gitlab:artifacts:check'
)
}.
to
output
(
/Checksum mismatch/
).
to_stdout
end
it
'errors out about missing checksum'
do
artifact
.
update_column
(
:file_sha256
,
nil
)
expect
{
run_rake_task
(
'gitlab:artifacts:check'
)
}.
to
output
(
/Checksum missing/
).
to_stdout
end
end
end
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录