index.md 9.9 KB
Newer Older
1 2 3 4 5 6
---
stage: Package
group: Package
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
---

7
# GitLab PyPi Repository
8

9 10
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3.
11 12 13 14 15 16 17 18 19 20

With the GitLab PyPi Repository, every project can have its own space to store PyPi packages.

The GitLab PyPi Repository works with:

- [pip](https://pypi.org/project/pip/)
- [twine](https://pypi.org/project/twine/)

## Setting up your development environment

21
You need a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/).
22 23 24 25 26

## Enabling the PyPi Repository

NOTE: **Note:**
This option is available only if your GitLab administrator has
27
[enabled support for the Package Registry](../../../administration/packages/index.md).
28

29
After the PyPi Repository is enabled, it is available for all new projects
30 31
by default. To enable it for existing projects, or if you want to disable it:

32
1. Navigate to your project's **Settings > General > Visibility, project features, permissions**.
33 34 35
1. Find the Packages feature and enable or disable it.
1. Click on **Save changes** for the changes to take effect.

36
You should then be able to see the **Packages & Registries** section on the left sidebar.
37

38 39
## Getting started

40
This section covers creating a new example PyPi package to upload. This is a
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157
quickstart to test out the **GitLab PyPi Registry**. If you already understand how
to build and publish your own packages, move on to the [next section](#adding-the-gitlab-pypi-repository-as-a-source).

### Create a project

Understanding how to create a full Python project is outside the scope of this
guide, but you can create a small package to test out the registry. Start by
creating a new directory called `MyPyPiPackage`:

```shell
mkdir MyPyPiPackage && cd MyPyPiPackage
```

After creating this, create another directory inside:

```shell
mkdir mypypipackage && cd mypypipackage
```

Create two new files inside this directory to set up the basic project:

```shell
touch __init__.py
touch greet.py
```

Inside `greet.py`, add the following code:

```python
def SayHello():
    print("Hello from MyPyPiPackage")
    return
```

Inside the `__init__.py` file, add the following:

```python
from .greet import SayHello
```

Now that the basics of our project is completed, we can test that the code runs.
Start the Python prompt inside your top `MyPyPiPackage` directory. Then run:

```python
>>> from mypypipackage import SayHello
>>> SayHello()
```

You should see an output similar to the following:

```plaintext
Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from mypypipackage import SayHello
>>> SayHello()
Hello from MyPyPiPackage
```

Once we've verified that the sample project is working as above, we can next
work on creating a package.

### Create a package

Inside your `MyPyPiPackage` directory, we need to create a `setup.py` file. Run
the following:

```shell
touch setup.py
```

This file contains all the information about our package. For more information
about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py).
For this guide, we don't need to extensively fill out this file, simply add the
below to your `setup.py`:

```python
import setuptools

setuptools.setup(
    name="mypypipackage",
    version="0.0.1",
    author="Example Author",
    author_email="author@example.com",
    description="A small example package",
    packages=setuptools.find_packages(),
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.6',
)
```

Save the file, then execute the setup like so:

```shell
python3 setup.py sdist bdist_wheel
```

If successful, you should be able to see the output in a newly created `dist`
folder. Run:

```shell
ls dist
```

And confirm your output matches the below:

```plaintext
mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz
```

Our package is now all set up and ready to be uploaded to the **GitLab PyPi
Package Registry**. Before we do so, we next need to set up authentication.

158 159
## Adding the GitLab PyPi Repository as a source

160 161
### Authenticating with a personal access token

162
You need the following:
163 164 165 166 167 168 169 170

- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication.
- A suitable name for your source.
- Your project ID which can be found on the home page of your project.

Edit your `~/.pypirc` file and add the following:

```ini
171 172 173 174
[distutils]
index-servers =
    gitlab

175 176 177 178 179 180
[gitlab]
repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi
username = __token__
password = <your personal access token>
```

181 182
### Authenticating with a deploy token

183
You need the following:
184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201

- A deploy token. You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the `read_package_registry` and/or `write_package_registry` scopes for repository authentication.
- A suitable name for your source.
- Your project ID which can be found on the home page of your project.

Edit your `~/.pypirc` file and add the following:

```ini
[distutils]
index-servers =
    gitlab

[gitlab]
repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi
username = <deploy token username>
password = <deploy token>
```

202 203 204 205 206
## Uploading packages

When uploading packages, note that:

- The maximum allowed size is 50 Megabytes.
207
- You cannot upload the same version of a package multiple times. If you try, you receive the error `Validation failed: File name has already been taken`.
208

209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228
### Ensure your version string is valid

If your version string (for example, `0.0.1`) is invalid, it will be rejected. GitLab uses the following regex to validate the version string.

```ruby
\A(?:
    v?
    (?:([0-9]+)!)?                                                 (?# epoch)
    ([0-9]+(?:\.[0-9]+)*)                                          (?# release segment)
    ([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)?  (?# pre-release)
    ((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))?       (?# post release)
    ([-_\.]?(dev)[-_\.]?([0-9]+)?)?                                (?# dev release)
    (?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))?                         (?# local version)
)\z}xi
```

You can play around with the regex and try your version strings on [this regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV).

For more details about the regex used, please check the [documentation here](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions))

229 230
### Upload packages with Twine

231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248
If you were following the guide above, then the `MyPyPiPackage` package should
be ready to be uploaded. Run the following command:

```shell
python3 -m twine upload --repository gitlab dist/*
```

If successful, you should see the following:

```plaintext
Uploading distributions to https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi
Uploading mypypipackage-0.0.1-py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s]
Uploading mypypipackage-0.0.1.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s]
```

This indicates that the package was uploaded successfully. You can then navigate
249
to your project's **Packages & Registries** page and see the uploaded packages.
250

251 252
If you did not follow the guide above, then you need to ensure your package
has been properly built and you [created a PyPi package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/).
253 254

You can then upload your package using the following command:
255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275

```shell
python -m twine upload --repository <source_name> dist/<package_file>
```

Where:

- `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`.
- `<source_name>` is the [source name used during setup](#adding-the-gitlab-pypi-repository-as-a-source).

## Install packages

Install the latest version of a package using the following command:

```shell
pip install --index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name>
```

Where:

- `<package_name>` is the package name.
276
- `<personal_access_token>` is a personal access token with the `read_api` scope.
277
- `<project_id>` is the project ID.
278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294

If you were following the guide above and want to test installing the
`MyPyPiPackage` package, you can run the following:

```shell
pip install mypypipackage --no-deps --index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple
```

This should result in the following:

```plaintext
Looking in indexes: https://__token__:****@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple
Collecting mypypipackage
  Downloading https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB)
Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.0.1
```