releasing_process_en.md 7.0 KB
Newer Older
T
typhoonzero 已提交
1 2
# PaddlePaddle Releasing Process

X
doc fix  
Xin Pan 已提交
3
PaddlePaddle manages its branches using Trunk Based Development, and [Semantic Versioning](http://semver.org/) as it's version number semantics.
T
typhoonzero 已提交
4 5 6

Each time we release a new PaddlePaddle version, we should follow the below steps:

X
Xin Pan 已提交
7
1. Create a new release branch from `develop`,named `release/[version]`. E.g.,`release/0.10.0`
X
clean  
Xin Pan 已提交
8
2. Create a new tag for the release branch, tag format: `version-rc.Patch`. E.g. the first tag is `0.10.0-rc0`
X
Xin Pan 已提交
9
3. New release branch normally doesn't accept new features or optimizations. QA will test on the release branch. Developer should develop based on `develop` branch.
X
clean  
Xin Pan 已提交
10
4. If QA or Developer find bugs. They should first fix and verify on `develop` branch. Then cherry-pick the fix to the release branch. Wait until the release branch is stable.
X
Xin Pan 已提交
11 12 13 14 15 16 17 18 19 20
5. If necessary, create a new tag on the relese branch, e.g. `0.10.0-rc1`. Involve more users to try it and repeat step 3-4.
6. After release branch is stable,Create the official release tag,such as `0.10.0`.
7. Release the python wheel package to pypi.
8. Update the docker image (More details below).

NOTE:

* bug fix should happen on `develop` branch, then cherry-pick to relese branch. Avoid developing directly on release branch.

* release normally only accept bug fixes. Don't add new features.
T
typhoonzero 已提交
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47


## Publish Wheel Packages to pypi

1. Use our [CI tool](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview)
   to build all wheel packages needed to publish. As shown in the following picture, choose a build
     version, click "..." button on the right side of "Run" button, and switch to the second tab in the
pop-up box, choose the current release branch and click "Run Build" button. You may repeat this
     step to start different versions of builds.
    <img src="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/ci_build_whl.png">
1. After the build succeeds, download the outputs under "Artifacts" including capi, `cp27m` and `cp27mu`.
1. Since pypi.python.org follows [PEP 513](https://www.python.org/dev/peps/pep-0513), before we
     upload the package using `twine`, we need to rename the package from `linux_x86_64` to
     `manylinux1_x86_64`.
1. Start the upload:
     ```
     cd build/python
     pip install twine
     twine upload dist/[package to upload]
     ```

* NOTE: We use a special Docker image to build our releases to support more Linux distributions, you can
  download it from https://hub.docker.com/r/paddlepaddle/paddle_manylinux_devel/tags/, or build it using
    scripts under `tools/manylinux1`.
* pypi does not allow overwrite the already uploaded version of wheel package, even if you delete the
  old version. you must change the version number before upload a new one.

T
typhoonzero 已提交
48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74
### Publish wheel Packages for MacOS

You need to build the binary wheel package for MacOS before publishing, to
make sure that the package can be used by many versions of MacOS
(10.11, 10.12, 10.13) and different python installs (python.org, homebrew, etc.),
you must build the package ***exactly*** following below steps:

Build steps:

1. install python from python.org downloads, and make sure it's currently in use
   in your system.
1. `export MACOSX_DEPLOYMENT_TARGET=10.11`, use `10.11` is enough for recent versions.
1. `git clone https://github.com/PaddlePaddle/Paddle.git && cd Paddle && mkdir build && cd build`
1. `cmake -DWITH_GPU=OFF -DWITH_MKL=OFF -DWITH_SYSTEM_BLAS=OFF  ..`, make sure the output of `cmake` command is using the correct python interpreter installed from python.org
1. `make -j`
1. `pip install delocate`
1. `mkdir fixed_wheel && delocate-wheel -w fixed_wheel python/dist/*.whl`

Then the whl under `fixed_wheel` is ready to upload.

Install steps:

1. run `pip install paddlepaddle...whl`
1. find the `libpython.dylib` that are currently in use:
    - for python.org package installs, do nothing.
    - for other python installs, find the path of `libpython*.dylib` and `export LD_LIBRARY_PATH=you path && DYLD_LIBRARY_PATH=your path`

T
typhoonzero 已提交
75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94
## Publish Docker Images

Our CI tool will push latest images to DockerHub, so we only need to push a version tag like:

```
docker pull [image]:latest
docker tag [image]:latest [image]:[version]
docker push [image]:[version]
```

Tags that need to be updated are:
* `[version]`: CPU only version image
* `[version]-openblas`: openblas version image
* `[version]-gpu`: GPU version(using CUDA 8.0 cudnn 5)
* `[version]-gpu-[cudaver]-[cudnnver]`: tag for different cuda, cudnn versions

You can then checkout the latest pushed tags at https://hub.docker.com/r/paddlepaddle/paddle/tags/.

## Branching Model

X
Xin Pan 已提交
95 96 97 98 99 100 101 102 103 104 105
PaddlePaddle uses [Trunk Based Development](https://trunkbaseddevelopment.com/) as our branching model.

* `develop` branch is used for development. Each comment to `develop` branc goes through unit tests and model regression tests.
* `release/[version]` branch is used for each release. Release branch is used for tests, bug fix and evetual release.
* `master` branch as been deprecated for historical reasons

* Developer's feature branch。
	* Developer's feature branch should sync with upstream `develop` branch.
	* Developer's feature branch should be forked from upstream `develop` branch.
	* After feature branch is ready, create a `Pull Request` against the Paddle repo and go through code review.
	   * In the review process, develop modify codes and push to their own feature branch.
T
typhoonzero 已提交
106 107 108

## PaddlePaddle Regression Test List

X
Xin Pan 已提交
109 110
TODO

T
typhoonzero 已提交
111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228
### All Chapters of PaddlePaddle Book

We need to guarantee that all the chapters of PaddlePaddle Book can run correctly. Including
V1 (`paddle_trainer` training) and V2 training and Fluid training.

<table>
<thead>
<tr>
<th></th>
<th>Linear Regression</th>
<th>Recognize Digits</th>
<th>Image Classification</th>
<th>Word2Vec</th>
<th>Personalized Recommendation</th>
<th>Sentiment Analysis</th>
<th>Semantic Role Labeling</th>
<th>Machine Translation</th>
</tr>
</thead>

<tbody>
<tr>
<td>API.V2 + Docker + GPU </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
</tr>

<tr>
<td> API.V2 + Docker + CPU </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
</tr>

<tr>
<td>`paddle_trainer` + Docker + GPU </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
</tr>

<tr>
<td>`paddle_trainer` + Docker + CPU </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
</tr>

<tr>
<td> API.V2 + Ubuntu + GPU</td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
</tr>

<tr>
<td>API.V2 + Ubuntu + CPU </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
</tr>

<tr>
<td> `paddle_trainer` + Ubuntu + GPU</td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
</tr>

<tr>
<td> `paddle_trainer` + Ubuntu + CPU</td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
<td>  </td>
<td> </td>
</tr>
</tbody>
</table>