README.md 20.1 KB
Newer Older
D
Douwe Maan 已提交
1 2
# Configuration of your builds with .gitlab-ci.yml

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
This document describes the usage of `.gitlab-ci.yml`, the file that is used by
GitLab Runner to manage your project's builds.

If you want a quick introduction to GitLab CI, follow our
[quick start guide](../quick_start/README.md).

---

<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [.gitlab-ci.yml](#gitlab-ci-yml)
    - [image and services](#image-and-services)
    - [before_script](#before_script)
    - [stages](#stages)
    - [types](#types)
    - [variables](#variables)
    - [cache](#cache)
        - [cache:key](#cache-key)
- [Jobs](#jobs)
    - [script](#script)
    - [stage](#stage)
    - [only and except](#only-and-except)
    - [tags](#tags)
    - [when](#when)
    - [artifacts](#artifacts)
        - [artifacts:name](#artifacts-name)
    - [dependencies](#dependencies)
- [Hidden jobs](#hidden-jobs)
- [Special YAML features](#special-yaml-features)
    - [Anchors](#anchors)
- [Validate the .gitlab-ci.yml](#validate-the-gitlab-ci-yml)
- [Skipping builds](#skipping-builds)
- [Examples](#examples)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

---

## .gitlab-ci.yml

45 46 47 48 49 50 51
From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML)
file (`.gitlab-ci.yml`) for the project configuration. It is placed in the root
of your repository and contains definitions of how your project should be built.

The YAML file defines a set of jobs with constraints stating when they should
be run. The jobs are defined as top-level elements with a name and always have
to contain the `script` clause:
D
Douwe Maan 已提交
52 53 54 55 56 57 58 59 60

```yaml
job1:
  script: "execute-script-for-job1"

job2:
  script: "execute-script-for-job2"
```

61 62 63 64 65
The above example is the simplest possible CI configuration with two separate
jobs, where each of the jobs executes a different command.

Of course a command can execute code directly (`./configure;make;make install`)
or run a script (`test.sh`) in the repository.
D
Douwe Maan 已提交
66

67
Jobs are used to create builds, which are then picked up by
68 69
[Runners](../runners/README.md) and executed within the environment of the
Runner. What is important, is that each job is run independently from each
70
other.
D
Douwe Maan 已提交
71

72 73
The YAML syntax allows for using more complex job specifications than in the
above example:
D
Douwe Maan 已提交
74 75

```yaml
J
James Lopez 已提交
76
image: ruby:2.1
D
Douwe Maan 已提交
77 78 79 80
services:
  - postgres

before_script:
F
frodsan 已提交
81
  - bundle install
D
Douwe Maan 已提交
82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97

stages:
  - build
  - test
  - deploy

job1:
  stage: build
  script:
    - execute-script-for-job1
  only:
    - master
  tags:
    - docker
```

98
There are a few reserved `keywords` that **cannot** be used as job names:
D
Douwe Maan 已提交
99

100
| Keyword       | Required | Description |
D
Douwe Maan 已提交
101
|---------------|----------|-------------|
102 103 104 105 106 107 108
| image         | no | Use docker image, covered in [Use Docker](../docker/README.md) |
| services      | no | Use docker services, covered in [Use Docker](../docker/README.md) |
| stages        | no | Define build stages |
| types         | no | Alias for `stages` |
| before_script | no | Define commands that run before each job's script |
| variables     | no | Define build variables |
| cache         | no | Define list of files that should be cached between subsequent runs |
D
Douwe Maan 已提交
109 110

### image and services
111 112 113

This allows to specify a custom Docker image and a list of services that can be
used for time of the build. The configuration of this feature is covered in
114
[a separate document](../docker/README.md).
D
Douwe Maan 已提交
115 116

### before_script
117 118 119

`before_script` is used to define the command that should be run before all
builds, including deploy builds. This can be an array or a multi-line string.
D
Douwe Maan 已提交
120 121

### stages
122

D
Douwe Maan 已提交
123 124 125 126 127 128
`stages` is used to define build stages that can be used by jobs.
The specification of `stages` allows for having flexible multi stage pipelines.

The ordering of elements in `stages` defines the ordering of builds' execution:

1. Builds of the same stage are run in parallel.
129 130
1. Builds of the next stage are run after the jobs from the previous stage
   complete successfully.
D
Douwe Maan 已提交
131 132

Let's consider the following example, which defines 3 stages:
133 134

```yaml
D
Douwe Maan 已提交
135 136 137 138 139 140 141
stages:
  - build
  - test
  - deploy
```

1. First all jobs of `build` are executed in parallel.
142 143 144
1. If all jobs of `build` succeed, the `test` jobs are executed in parallel.
1. If all jobs of `test` succeed, the `deploy` jobs are executed in parallel.
1. If all jobs of `deploy` succeed, the commit is marked as `success`.
145 146
1. If any of the previous jobs fails, the commit is marked as `failed` and no
   jobs of further stage are executed.
D
Douwe Maan 已提交
147 148 149

There are also two edge cases worth mentioning:

150 151
1. If no `stages` is defined in `.gitlab-ci.yml`, then by default the `build`,
   `test` and `deploy` are allowed to be used as job's stage by default.
D
Douwe Maan 已提交
152 153 154
2. If a job doesn't specify `stage`, the job is assigned the `test` stage.

### types
155

D
Douwe Maan 已提交
156 157 158 159
Alias for [stages](#stages).

### variables

160 161
>**Note:**
Introduced in GitLab Runner v0.5.0.
162 163 164 165

GitLab CI allows you to add to `.gitlab-ci.yml` variables that are set in build
environment. The variables are stored in the git repository and are meant to
store non-sensitive project configuration, for example:
D
Douwe Maan 已提交
166 167 168 169 170 171 172 173

```yaml
variables:
  DATABASE_URL: "postgres://postgres@postgres/my_database"
```

These variables can be later used in all executed commands and scripts.

174 175
The YAML-defined variables are also set to all created service containers,
thus allowing to fine tune them.
D
Douwe Maan 已提交
176

177 178
### cache

179 180 181
>**Note:**
Introduced in GitLab Runner v0.7.0.

182
`cache` is used to specify a list of files and directories which should be
183 184 185
cached between builds.

**By default the caching is enabled per-job and per-branch.**
186 187 188

If `cache` is defined outside the scope of the jobs, it means it is set
globally and all jobs will use its definition.
189

190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222
Cache all files in `binaries` and `.config`:

```yaml
rspec:
  script: test
  cache:
    paths:
    - binaries/
    - .config
```

Cache all Git untracked files:

```yaml
rspec:
  script: test
  cache:
    untracked: true
```

Cache all Git untracked files and files in `binaries`:

```yaml
rspec:
  script: test
  cache:
    untracked: true
    paths:
    - binaries/
```

Locally defined cache overwrites globally defined options. This will cache only
`binaries/`:
223 224

```yaml
225 226
cache:
  paths:
227 228 229 230 231 232 233
  - my/files

rspec:
  script: test
  cache:
    paths:
    - binaries/
234 235
```

236 237 238
The cache is provided on best effort basis, so don't expect that cache will be
always present. For implementation details please check GitLab Runner.

239 240
#### cache:key

241 242
>**Note:**
Introduced in GitLab Runner v1.0.0.
243 244 245 246 247

The `key` directive allows you to define the affinity of caching
between jobs, allowing to have a single cache for all jobs,
cache per-job, cache per-branch or any other way you deem proper.

248 249
This allows you to fine tune caching, allowing you to cache data between
different jobs or even different branches.
250

251 252 253 254 255
The `cache:key` variable can use any of the [predefined variables](../variables/README.md).

---

**Example configurations**
256 257 258

To enable per-job caching:

259 260 261 262 263
```yaml
cache:
  key: "$CI_BUILD_NAME"
  untracked: true
```
264 265 266

To enable per-branch caching:

267 268 269 270 271
```yaml
cache:
  key: "$CI_BUILD_REF_NAME"
  untracked: true
```
272 273 274

To enable per-job and per-branch caching:

275 276 277 278 279
```yaml
cache:
  key: "$CI_BUILD_NAME/$CI_BUILD_REF_NAME"
  untracked: true
```
280 281 282

To enable per-branch and per-stage caching:

283 284 285 286 287
```yaml
cache:
  key: "$CI_BUILD_STAGE/$CI_BUILD_REF_NAME"
  untracked: true
```
288

289 290
If you use **Windows Batch** to run your shell scripts you need to replace
`$` with `%`:
291

292 293 294 295 296
```yaml
cache:
  key: "%CI_BUILD_STAGE%/%CI_BUILD_REF_NAME%"
  untracked: true
```
297

D
Douwe Maan 已提交
298
## Jobs
299 300 301 302

`.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job
must have a unique name, which is not one of the Keywords mentioned above.
A job is defined by a list of parameters that define the build behavior.
D
Douwe Maan 已提交
303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319

```yaml
job_name:
  script:
    - rake spec
    - coverage
  stage: test
  only:
    - master
  except:
    - develop
  tags:
    - ruby
    - postgres
  allow_failure: true
```

320
| Keyword       | Required | Description |
D
Douwe Maan 已提交
321
|---------------|----------|-------------|
322
| script        | yes | Defines a shell script which is executed by Runner |
P
Pat Turner 已提交
323 324
| image         | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| services      | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
325
| stage         | no | Defines a build stage (default: `test`) |
326 327 328
| type          | no | Alias for `stage` |
| only          | no | Defines a list of git refs for which build is created |
| except        | no | Defines a list of git refs for which build is not created |
329
| tags          | no | Defines a list of tags which are used to select Runner |
330 331
| allow_failure | no | Allow build to fail. Failed build doesn't contribute to commit status |
| when          | no | Define when to run build. Can be `on_success`, `on_failure` or `always` |
332
| dependencies  | no | Define other builds that a build depends on so that you can pass artifacts between them|
333 334
| artifacts     | no | Define list build artifacts |
| cache         | no | Define list of files that should be cached between subsequent runs |
D
Douwe Maan 已提交
335 336

### script
337

338
`script` is a shell script which is executed by the Runner. For example:
D
Douwe Maan 已提交
339 340 341 342 343 344 345

```yaml
job:
  script: "bundle exec rspec"
```

This parameter can also contain several commands using an array:
346

D
Douwe Maan 已提交
347 348 349 350 351 352 353 354
```yaml
job:
  script:
    - uname -a
    - bundle exec rspec
```

### stage
355 356 357 358

`stage` allows to group build into different stages. Builds of the same `stage`
are executed in `parallel`. For more info about the use of `stage` please check
[stages](#stages).
D
Douwe Maan 已提交
359 360 361

### only and except

362 363
`only` and `except` are two parameters that set a refs policy to limit when
jobs are built:
D
Douwe Maan 已提交
364

365 366 367 368 369 370 371 372 373 374
1. `only` defines the names of branches and tags for which the job will be
    built.
2. `except` defines the names of branches and tags for which the job will
    **not** be built.

There are a few rules that apply to the usage of refs policy:

* `only` and `except` are inclusive. If both `only` and `except` are defined
   in a job specification, the ref is filtered by `only` and `except`.
* `only` and `except` allow the use of regular expressions.
375
* `only` and `except` allow the use of special keywords: `branches`, `tags`, and `triggers`.
376 377 378 379 380
* `only` and `except` allow to specify a repository path to filter jobs for
   forks.

In the example below, `job` will run only for refs that start with `issue-`,
whereas all branches will be skipped.
D
Douwe Maan 已提交
381 382 383

```yaml
job:
384
  # use regexp
D
Douwe Maan 已提交
385
  only:
386 387
    - /^issue-.*$/
  # use special keyword
D
Douwe Maan 已提交
388
  except:
389
    - branches
D
Douwe Maan 已提交
390 391
```

392 393 394 395 396 397 398 399 400 401 402
In this example, `job` will run only for refs that are tagged, or if a build is explicitly requested
via an API trigger.

```yaml
job:
  # use special keywords
  only:
    - tags
    - triggers
```

403 404
The repository path can be used to have jobs executed only for the parent
repository and not forks:
405 406 407 408 409 410 411 412

```yaml
job:
  only:
    - branches@gitlab-org/gitlab-ce
  except:
    - master@gitlab-org/gitlab-ce
```
413 414 415

The above example will run `job` for all branches on `gitlab-org/gitlab-ce`,
except master.
416

D
Douwe Maan 已提交
417 418
### tags

419
`tags` is used to select specific Runners from the list of all Runners that are
420
allowed to run this project.
D
Douwe Maan 已提交
421

422
During the registration of a Runner, you can specify the Runner's tags, for
423 424
example `ruby`, `postgres`, `development`.

425
`tags` allow you to run builds with Runners that have the specified tags
426 427 428
assigned to them:

```yaml
D
Douwe Maan 已提交
429 430 431 432 433 434
job:
  tags:
    - ruby
    - postgres
```

435
The specification above, will make sure that `job` is built by a Runner that
436
has both `ruby` AND `postgres` tags defined.
D
Douwe Maan 已提交
437

438
### when
439 440 441

`when` is used to implement jobs that are run in case of failure or despite the
failure.
442

R
Robert Speicher 已提交
443 444
`when` can be set to one of the following values:

445 446 447 448
1. `on_success` - execute build only when all builds from prior stages
    succeeded. This is the default.
1. `on_failure` - execute build only when at least one build from prior stages
    failed.
449 450
1. `always` - execute build despite the status of builds from prior stages.

451 452 453
For example:

```yaml
454 455 456 457 458 459 460
stages:
- build
- cleanup_build
- test
- deploy
- cleanup

461
build_job:
462 463 464 465
  stage: build
  script:
  - make build

466
cleanup_build_job:
467 468 469 470 471
  stage: cleanup_build
  script:
  - cleanup build when failed
  when: on_failure

472
test_job:
473 474 475 476
  stage: test
  script:
  - make test

477
deploy_job:
478 479 480 481
  stage: deploy
  script:
  - make deploy

482
cleanup_job:
483 484 485 486 487 488 489
  stage: cleanup
  script:
  - cleanup after builds
  when: always
```

The above script will:
490 491 492

1. Execute `cleanup_build_job` only when `build_job` fails
2. Always execute `cleanup_job` as the last step in pipeline.
493

K
Kamil Trzcinski 已提交
494 495
### artifacts

496 497 498
>**Notes:**
>
> - Introduced in GitLab Runner v0.7.0 for non-Windows platforms.
499
> - Windows support was added in GitLab Runner v.1.0.0.
500 501
> - Currently not all executors are supported.
> - Build artifacts are only collected for successful builds.
502

503
`artifacts` is used to specify list of files and directories which should be
504 505 506 507
attached to build after success. To pass artifacts between different builds,
see [dependencies](#dependencies).

Below are some examples.
508

509
Send all files in `binaries` and `.config`:
510

511 512 513 514 515 516
```yaml
artifacts:
  paths:
  - binaries/
  - .config
```
517

518
Send all Git untracked files:
519

520 521 522 523 524
```yaml
artifacts:
  untracked: true
```

525
Send all Git untracked files and files in `binaries`:
K
Kamil Trzcinski 已提交
526

527 528 529 530 531 532
```yaml
artifacts:
  untracked: true
  paths:
  - binaries/
```
K
Kamil Trzcinski 已提交
533

534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555
You may want to create artifacts only for tagged releases to avoid filling the
build server storage with temporary build artifacts.

Create artifacts only for tags (`default-job` will not create artifacts):

```yaml
default-job:
  script:
    - mvn test -U
  except:
    - tags

release-job:
  script:
    - mvn package -U
  artifacts:
    paths:
    - target/*.war
  only:
    - tags
```

556 557
The artifacts will be sent to GitLab after a successful build and will
be available for download in the GitLab UI.
K
Kamil Trzcinski 已提交
558

559 560
#### artifacts:name

561 562
>**Note:**
Introduced in GitLab 8.6 and GitLab Runner v1.1.0.
563

A
Achilleas Pipinellis 已提交
564 565 566 567
The `name` directive allows you to define the name of the created artifacts
archive. That way, you can have a unique name of every archive which could be
useful when you'd like to download the archive from GitLab. The `artifacts:name`
variable can make use of any of the [predefined variables](../variables/README.md).
568 569 570 571 572

---

**Example configurations**

A
Achilleas Pipinellis 已提交
573
To create an archive with a name of the current build:
574 575 576 577 578 579 580

```yaml
job:
  artifacts:
    name: "$CI_BUILD_NAME"
```

A
Achilleas Pipinellis 已提交
581 582
To create an archive with a name of the current branch or tag including only
the files that are untracked by Git:
583 584 585 586 587 588 589 590

```yaml
job:
   artifacts:
     name: "$CI_BUILD_REF_NAME"
     untracked: true
```

A
Achilleas Pipinellis 已提交
591 592
To create an archive with a name of the current build and the current branch or
tag including only the files that are untracked by Git:
593 594 595 596 597 598 599 600

```yaml
job:
  artifacts:
    name: "${CI_BUILD_NAME}_${CI_BUILD_REF_NAME}"
    untracked: true
```

A
Achilleas Pipinellis 已提交
601
To create an archive with a name of the current [stage](#stages) and branch name:
602 603 604 605 606 607 608 609

```yaml
job:
  artifacts:
    name: "${CI_BUILD_STAGE}_${CI_BUILD_REF_NAME}"
    untracked: true
```

A
Achilleas Pipinellis 已提交
610 611
---

612 613 614 615 616 617 618 619 620 621
If you use **Windows Batch** to run your shell scripts you need to replace
`$` with `%`:

```yaml
job:
  artifacts:
    name: "%CI_BUILD_STAGE%_%CI_BUILD_REF_NAME%"
    untracked: true
```

622 623
### dependencies

624 625
>**Note:**
Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
626

627 628
This feature should be used in conjunction with [`artifacts`](#artifacts) and
allows you to define the artifacts to pass between different builds.
629

630
Note that `artifacts` from previous [stages](#stages) are passed by default.
631

632
To use this feature, define `dependencies` in context of the job and pass
633
a list of all previous builds from which the artifacts should be downloaded.
634 635 636 637
You can only define builds from stages that are executed before the current one.
An error will be shown if you define builds from the current stage or next ones.

---
638

639 640 641 642 643 644 645
In the following example, we define two jobs with artifacts, `build:osx` and
`build:linux`. When the `test:osx` is executed, the artifacts from `build:osx`
will be downloaded and extracted in the context of the build. The same happens
for `test:linux` and artifacts from `build:linux`.

The job `deploy` will download artifacts from all previous builds because of
the [stage](#stages) precedence:
646

647
```yaml
648 649
build:osx:
  stage: build
650
  script: make build:osx
651 652 653
  artifacts:
    paths:
    - binaries/
654

655 656
build:linux:
  stage: build
657
  script: make build:linux
658 659 660 661 662 663
  artifacts:
    paths:
    - binaries/

test:osx:
  stage: test
664
  script: make test:osx
665 666 667 668 669
  dependencies:
  - build:osx

test:linux:
  stage: test
670
  script: make test:linux
671 672 673 674 675
  dependencies:
  - build:linux

deploy:
  stage: deploy
676
  script: make deploy
677 678
```

A
Achilleas Pipinellis 已提交
679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696
## Hidden jobs

>**Note:**
Introduced in GitLab 8.6 and GitLab Runner v1.1.1.

Jobs that start with a dot (`.`) will be not processed by GitLab CI. You can
use this feature to ignore jobs, or use the
[special YAML features](#special-yaml-features) and transform the hidden jobs
into templates.

In the following example, `.job_name` will be ignored:

```yaml
.job_name:
  script:
    - rake spec
```

697
## Special YAML features
698

699 700 701
It's possible to use special YAML features like anchors (`&`), aliases (`*`)
and map merging (`<<`), which will allow you to greatly reduce the complexity
of `.gitlab-ci.yml`.
702

703
Read more about the various [YAML features](https://learnxinyminutes.com/docs/yaml/).
704

705 706 707 708 709 710 711 712 713 714 715 716 717
### Anchors

>**Note:**
Introduced in GitLab 8.6 and GitLab Runner v1.1.1.

YAML also has a handy feature called 'anchors', which let you easily duplicate
content across your document. Anchors can be used to duplicate/inherit
properties, and is a perfect example to be used with [hidden jobs](#hidden-jobs)
to provide templates for your jobs.

The following example uses anchors and map merging. It will create two jobs,
`test1` and `test2`, that will inherit the parameters of `.job_template`, each
having their own custom `script` defined:
718 719

```yaml
720
.job_template: &job_definition  # Hidden job that defines an anchor named 'job_definition'
721 722 723 724 725 726
  image: ruby:2.1
  services:
    - postgres
    - redis

test1:
727
  <<: *job_definition           # Merge the contents of the 'job_definition' alias
728
  script:
729
    - test1 project
730 731

test2:
732
  <<: *job_definition           # Merge the contents of the 'job_definition' alias
733
  script:
734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762
    - test2 project
```

`&` sets up the name of the anchor (`job_definition`), `<<` means "merge the
given hash into the current one", and `*` includes the named anchor
(`job_definition` again). The expanded version looks like this:

```yaml
.job_template:
  image: ruby:2.1
  services:
    - postgres
    - redis

test1:
  image: ruby:2.1
  services:
    - postgres
    - redis
  script:
    - test1 project

test2:
  image: ruby:2.1
  services:
    - postgres
    - redis
  script:
    - test2 project
763 764
```

765 766 767 768
Let's see another one example. This time we will use anchors to define two sets
of services. This will create two jobs, `test:postgres` and `test:mysql`, that
will share the `script` directive defined in `.job_template`, and the `services`
directive defined in `.postgres_services` and `.mysql_services` respectively:
769 770 771 772 773 774 775 776 777 778 779

```yaml
.job_template: &job_definition
  script:
    - test project

.postgres_services:
  services: &postgres_definition
    - postgres
    - ruby

780
.mysql_services:
781 782 783 784 785 786 787 788 789 790 791 792 793
  services: &mysql_definition
    - mysql
    - ruby

test:postgres:
  << *job_definition
  services: *postgres_definition

test:mysql:
  << *job_definition
  services: *mysql_definition
```

794
The expanded version looks like this:
795

796 797 798 799
```yaml
.job_template:
  script:
    - test project
800

801 802 803 804
.postgres_services:
  services:
    - postgres
    - ruby
805

806 807 808 809 810 811
.mysql_services:
  services:
    - mysql
    - ruby

test:postgres:
812
  script:
813 814 815 816 817 818 819 820 821 822 823
    - test project
  services:
    - postgres
    - ruby

test:mysql:
  script:
    - test project
  services:
    - mysql
    - ruby
824 825
```

826
You can see that the hidden jobs are conveniently used as templates.
827

D
Douwe Maan 已提交
828
## Validate the .gitlab-ci.yml
829

D
Douwe Maan 已提交
830
Each instance of GitLab CI has an embedded debug tool called Lint.
831
You can find the link under `/ci/lint` of your gitlab instance.
D
Douwe Maan 已提交
832 833

## Skipping builds
834 835 836

If your commit message contains `[ci skip]`, the commit will be created but the
builds will be skipped.
A
Achilleas Pipinellis 已提交
837 838 839 840 841 842 843

## Examples

Visit the [examples README][examples] to see a list of examples using GitLab
CI with various languages.

[examples]: ../examples/README.md