511.md 10.3 KB
Newer Older
Lab机器人's avatar
readme  
Lab机器人 已提交
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195
# Clean up

> 原文:[https://docs.gitlab.com/ee/raketasks/cleanup.html](https://docs.gitlab.com/ee/raketasks/cleanup.html)

*   [Remove unreferenced LFS files](#remove-unreferenced-lfs-files)
    *   [Remove unreferenced LFS files immediately](#remove-unreferenced-lfs-files-immediately)
*   [Clean up project upload files](#clean-up-project-upload-files)
    *   [Clean up project upload files from filesystem](#clean-up-project-upload-files-from-filesystem)
    *   [Clean up project upload files from object storage](#clean-up-project-upload-files-from-object-storage)
*   [Remove orphan artifact files](#remove-orphan-artifact-files)
*   [Remove expired ActiveSession lookup keys](#remove-expired-activesession-lookup-keys)
*   [Container Registry garbage collection](#container-registry-garbage-collection)

# Clean up[](#clean-up-core-only "Permalink")

GitLab 提供 Rake 任务来清理 GitLab 实例.

## Remove unreferenced LFS files[](#remove-unreferenced-lfs-files "Permalink")

在 GitLab 12.10 中[引入](https://gitlab.com/gitlab-org/gitlab/-/issues/36628) .

**危险:**不要在 GitLab 升级后的 12 小时内运行此程序. 这是为了确保所有后台迁移均已完成,否则可能导致数据丢失.

当您从存储库的历史记录中删除 LFS 文件时,它们将变成孤立文件并继续占用磁盘空间. 使用此 Rake 任务,您可以从数据库中删除无效的引用,这将允许对 LFS 文件进行垃圾回收.

例如:

```
# omnibus-gitlab
sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_PATH="gitlab-org/gitlab-foss"

# installation from source
bundle exec rake gitlab:cleanup:orphan_lfs_file_references RAILS_ENV=production PROJECT_PATH="gitlab-org/gitlab-foss" 
```

您也可以使用`PROJECT_ID`而不是`PROJECT_PATH`指定项目.

例如:

```
$ sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_PATH="gitlab-org/gitlab-foss"
I, [2019-12-13T16:35:31.764962 #82356]  INFO -- :  Looking for orphan LFS files for project GitLab Org / GitLab Foss
I, [2019-12-13T16:35:31.923659 #82356]  INFO -- :  Removed invalid references: 12 
```

默认情况下,此任务不会删除任何内容,但会显示它可以删除多少个文件引用. 如果您确实要删除引用,请运行`DRY_RUN=false`的命令. 您还可以使用`LIMIT={number}`参数来限制已删除引用的数量.

请注意,此 Rake 任务仅删除对 LFS 文件的引用. 未引用的 LFS 文件将在以后(每天一次)进行垃圾收集. 如果您需要立即进行垃圾回收,请运行以下描述的`rake gitlab:cleanup:orphan_lfs_files` .

### Remove unreferenced LFS files immediately[](#remove-unreferenced-lfs-files-immediately "Permalink")

在 GitLab 12.10 中[引入](https://gitlab.com/gitlab-org/gitlab/-/issues/36628) .

每天都会删除未引用的 LFS 文件,但是如果需要,您可以立即删除它们. 例如:

```
# omnibus-gitlab
sudo gitlab-rake gitlab:cleanup:orphan_lfs_files

# installation from source
bundle exec rake gitlab:cleanup:orphan_lfs_files 
```

输出示例:

```
$ sudo gitlab-rake gitlab:cleanup:orphan_lfs_files
I, [2020-01-08T20:51:17.148765 #43765]  INFO -- : Removed unreferenced LFS files: 12 
```

## Clean up project upload files[](#clean-up-project-upload-files "Permalink")

如果项目上传文件在 GitLab 数据库中不存在,请清理它们.

### Clean up project upload files from filesystem[](#clean-up-project-upload-files-from-filesystem "Permalink")

在 GitLab 11.2 中[引入](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20863) .

如果本地项目上传文件在 GitLab 数据库中不存在,请清理它们. 如果该任务可以找到其项目,则该任务将尝试修复该文件,否则,它将文件移至丢失并找到的目录.

```
# omnibus-gitlab
sudo gitlab-rake gitlab:cleanup:project_uploads

# installation from source
bundle exec rake gitlab:cleanup:project_uploads RAILS_ENV=production 
```

输出示例:

```
$ sudo gitlab-rake gitlab:cleanup:project_uploads

I, [2018-07-27T12:08:27.671559 #89817]  INFO -- : Looking for orphaned project uploads to clean up. Dry run...
D, [2018-07-27T12:08:28.293568 #89817] DEBUG -- : Processing batch of 500 project upload file paths, starting with /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out
I, [2018-07-27T12:08:28.689869 #89817]  INFO -- : Can move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/test.out
I, [2018-07-27T12:08:28.755624 #89817]  INFO -- : Can fix /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/qux/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
I, [2018-07-27T12:08:28.760257 #89817]  INFO -- : Can move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png
I, [2018-07-27T12:08:28.764470 #89817]  INFO -- : To cleanup these files run this command with DRY_RUN=false

$ sudo gitlab-rake gitlab:cleanup:project_uploads DRY_RUN=false I, [2018-07-27T12:08:32.944414 #89936]  INFO -- : Looking for orphaned project uploads to clean up...
D, [2018-07-27T12:08:33.293568 #89817] DEBUG -- : Processing batch of 500 project upload file paths, starting with /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out
I, [2018-07-27T12:08:33.689869 #89817]  INFO -- : Did move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/test.out
I, [2018-07-27T12:08:33.755624 #89817]  INFO -- : Did fix /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/qux/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
I, [2018-07-27T12:08:33.760257 #89817]  INFO -- : Did move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png 
```

### Clean up project upload files from object storage[](#clean-up-project-upload-files-from-object-storage "Permalink")

在 GitLab 11.2 中[引入](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20918) .

如果对象存储库上载文件在 GitLab 数据库中不存在,请将其移动到丢失和找到的目录中.

```
# omnibus-gitlab
sudo gitlab-rake gitlab:cleanup:remote_upload_files

# installation from source
bundle exec rake gitlab:cleanup:remote_upload_files RAILS_ENV=production 
```

Example output:

```
$ sudo gitlab-rake gitlab:cleanup:remote_upload_files

I, [2018-08-02T10:26:13.995978 #45011]  INFO -- : Looking for orphaned remote uploads to remove. Dry run...
I, [2018-08-02T10:26:14.120400 #45011]  INFO -- : Can be moved to lost and found: @hashed/6b/DSC_6152.JPG
I, [2018-08-02T10:26:14.120482 #45011]  INFO -- : Can be moved to lost and found: @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg
I, [2018-08-02T10:26:14.120634 #45011]  INFO -- : To cleanup these files run this command with DRY_RUN=false 
```

```
$ sudo gitlab-rake gitlab:cleanup:remote_upload_files DRY_RUN=false I, [2018-08-02T10:26:47.598424 #45087]  INFO -- : Looking for orphaned remote uploads to remove...
I, [2018-08-02T10:26:47.753131 #45087]  INFO -- : Moved to lost and found: @hashed/6b/DSC_6152.JPG -> lost_and_found/@hashed/6b/DSC_6152.JPG
I, [2018-08-02T10:26:47.764356 #45087]  INFO -- : Moved to lost and found: @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg -> lost_and_found/@hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg 
```

## Remove orphan artifact files[](#remove-orphan-artifact-files "Permalink")

版本历史

*   在 GitLab 12.1 中[引入](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/29681) .
*   在 GitLab 12.10 中[修复](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28023)[`ionice`支持](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28023) .

如果发现磁盘上的作业工件文件多于应有的数量,则可以运行:

```
sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files 
```

该命令:

*   扫描整个工件文件夹.
*   检查哪些文件在数据库中仍然有记录.
*   如果找不到数据库记录,那么将从磁盘中删除该文件.

默认情况下,此任务不会删除任何内容,但会显示它可以删除的内容. 如果您实际上要删除文件,请运行`DRY_RUN=false`的命令:

```
sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files DRY_RUN=false 
```

您还可以使用`LIMIT`要删除的文件数:

```
sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files LIMIT=100 
```

这只会从磁盘上删除多达 100 个文件. 您可以使用此功能删除一小集以进行测试.

如果提供`DEBUG=1` ,则将看到被检测为孤立文件的每个文件的完整路径.

如果安装了`ionice` ,则任务将使用它来确保命令不会在磁盘上造成太大的负载. 您可以使用`NICENESS`配置`NICENESS`级别. 以下是有效水平,但请务必咨询`man 1 ionice` .

*   `0` or `None`
*   `1` or `Realtime`
*   `2` or `Best-effort` (default)
*   `3` or `Idle`

## Remove expired ActiveSession lookup keys[](#remove-expired-activesession-lookup-keys "Permalink")

在 GitLab 12.2 中[引入](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30668) .

```
# omnibus-gitlab
sudo gitlab-rake gitlab:cleanup:sessions:active_sessions_lookup_keys

# installation from source
bundle exec rake gitlab:cleanup:sessions:active_sessions_lookup_keys RAILS_ENV=production 
```

## Container Registry garbage collection[](#container-registry-garbage-collection "Permalink")

容器注册表可以使用大量的磁盘空间. 为了清除未使用的层,注册表包括一个[垃圾收集命令](../administration/packages/container_registry.html#container-registry-garbage-collection) .