wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-06-30 19:08:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update Release Process documentation (#4670)

Browse Source
This commit is contained in:
Devin Petersohn
2019-04-25 00:05:19 -07:00
committed by Robert Nishihara
parent c26f24ab9f
commit fb2655fa93
2 changed files with 70 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ci/long_running_tests/config.yaml
+1 -1
View File
@@ -51,7 +51,7 @@ setup_commands:
# - git clone https://github.com/ray-project/ray || true
# - cd ray/python; git checkout master; git pull; pip install -e . --verbose
# Install nightly Ray wheels.
- pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-0.7.0.dev2-cp36-cp36m-manylinux1_x86_64.whl
- pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/<<RAY_RELEASE_HASH>>/ray-<<RAY_RELEASE_VERSION>>-cp36-cp36m-manylinux1_x86_64.whl
- pip install ray[rllib] ray[debug] tensorflow
- pip install -U dask # fix error importing lz4
dev/RELEASE_PROCESS.rst
+69 -29
View File
@@ -3,46 +3,72 @@ Release Process
This document describes the process for creating new releases.
1. First, you should build wheels that you'd like to use for testing. That can
be done by following the `documentation for building wheels`_.
2. **Testing:** Before a release is created, significant testing should be done.
Run the script `ci/stress_tests/run_stress_tests.sh`_ and
`ci/stress_tests/run_application_stress_tests.sh`_ and make sure it
passes. *And make sure it is testing the right version of Ray!* This will use
the autoscaler to start a bunch of machines and run some tests. Any new
stress tests should be added to this script so that they will be run
automatically for future release testing.
3. **Libraries:** Make sure that the libraries (e.g., RLlib, Tune, SGD) are in a
releasable state. TODO(rkn): These libraries should be tested automatically
by the script above, but they aren't yet.
4. **Increment the Python version:** Create a PR that increments the Python
1. **Increment the Python version:** Create a PR that increments the Python
package version. See `this example`_.
5. **Create a GitHub release:** Create a GitHub release through the `GitHub
website`_. The release should be created at the commit from the previous
step. This should include **release notes**. Copy the style and formatting
used by previous releases. Use the following to get started:
2. **Download the Travis-built wheels:** Once Travis has completed the tests,
the wheels from this commit can be downloaded from S3 to do testing, etc.
The URL is structured like this:
``https://s3-us-west-2.amazonaws.com/ray-wheels/<hash>/<wheel-name>``
where ``<hash>`` is replaced by the ID of the commit and the ``<version>``
is the incremented version from the previous step. The ``<wheel-name>`` can
be determined by looking at the OS/Version matrix in the documentation_.
3. **Create a release branch:** This branch should also have the same commit ID as the
previous two steps. In order to create the branch, locally checkout the commit ID
i.e. ``git checkout <hash>``. Then checkout a new branch of the format
``releases/<release_number>``. The release number must match the increment in
the first step. Then push that branch to the ray repo:
``git push upstream releases/<release_number>``.
4. **Testing:** Before a release is created, significant testing should be done.
Run the scripts `ci/stress_tests/run_stress_tests.sh`_ and
`ci/stress_tests/run_application_stress_tests.sh`_ and make sure they
pass. You **MUST** modify the autoscaler config file and replace
``<<RAY_RELEASE_HASH>>`` and ``<<RAY_RELEASE_VERSION>>`` with the appropriate
values to test the correct wheels. This will use the autoscaler to start a bunch of
machines and run some tests. Any new stress tests should be added to this
script so that they will be run automatically for future release testing.
5. **Resolve release-blockers:** Should any release blocking issues arise,
there are two ways these issues are resolved: A PR to patch the issue or a
revert commit that removes the breaking change from the release. In the case
of a PR, that PR should be created against master. Once it is merged, the
release master should ``git cherry-pick`` the commit to the release branch.
If the decision is to revert a commit that caused the release blocker, the
release master should ``git revert`` the commit to be reverted on the
release branch. Push these changes directly to the release branch.
6. **Download all the wheels:** Now the release is ready to begin final
testing. The wheels are automatically uploaded to S3, even on the release
branch. The wheels can ``pip install``ed from the following URLs:
.. code-block:: bash
git pull origin master --tags
git log $(git describe --tags --abbrev=0)..HEAD --pretty=format:"%s" | sort
export RAY_HASH=... # e.g., 618147f57fb40368448da3b2fb4fd213828fa12b
export RAY_VERSION=... # e.g., 0.6.6
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/$RAY_HASH/ray-$RAY_VERSION-cp27-cp27mu-manylinux1_x86_64.whl
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/$RAY_HASH/ray-$RAY_VERSION-cp35-cp35m-manylinux1_x86_64.whl
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/$RAY_HASH/ray-$RAY_VERSION-cp36-cp36m-manylinux1_x86_64.whl
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/$RAY_HASH/ray-$RAY_VERSION-cp37-cp37m-manylinux1_x86_64.whl
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/$RAY_HASH/ray-$RAY_VERSION-cp27-cp27m-macosx_10_6_intel.whl
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/$RAY_HASH/ray-$RAY_VERSION-cp35-cp35m-macosx_10_6_intel.whl
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/$RAY_HASH/ray-$RAY_VERSION-cp36-cp36m-macosx_10_6_intel.whl
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/$RAY_HASH/ray-$RAY_VERSION-cp37-cp37m-macosx_10_6_intel.whl
6. **Python wheels:** The Python wheels will automatically be built on Travis
and uploaded to the ``ray-wheels`` S3 bucket. Download these wheels (e.g.,
using ``wget``) and install them with ``pip`` and run some simple Ray scripts
to verify that they work. Find `these wheels here`_.
7. **Final Testing:** Send a link to the wheels to the other contributors and
core members of the Ray project. Make sure the wheels are tested on Ubuntu,
Mac OSX 10.12, and Mac OSX 10.13+. This testing should verify that the
wheels are correct and that all release blockers have been resolved. Should
a new release blocker be found, repeat steps 5-7.
7. **Upload to PyPI Test:** Upload the wheels to the PyPI test site using
8. **Upload to PyPI Test:** Upload the wheels to the PyPI test site using
``twine`` (ask Robert to add you as a maintainer to the PyPI project). You'll
need to run a command like
.. code-block:: bash
twine upload --repository-url https://test.pypi.org/legacy/ ray/.whl/*
twine upload --repository-url https://test.pypi.org/legacy/ray/.whl/*
assuming that you've downloaded the wheels from the ``ray-wheels`` S3 bucket
and put them in ``ray/.whl``, that you've installed ``twine`` through
@@ -62,7 +88,7 @@ This document describes the process for creating new releases.
Do this at least for MacOS and for Linux, as well as for Python 2 and Python
3. Also do this for different versions of MacOS.
8. **Upload to PyPI:** Now that you've tested the wheels on the PyPI test
9. **Upload to PyPI:** Now that you've tested the wheels on the PyPI test
repository, they can be uploaded to the main PyPI repository. Be careful,
**it will not be possible to modify wheels once you upload them**, so any
mistake will require a new release. You can upload the wheels with a command
@@ -81,6 +107,20 @@ This document describes the process for creating new releases.
finds the correct Ray version, and successfully runs some simple scripts on
both MacOS and Linux as well as Python 2 and Python 3.
10. **Create a GitHub release:** Create a GitHub release through the `GitHub website`_.
The release should be created at the commit from the previous
step. This should include **release notes**. Copy the style and formatting
used by previous releases. Create a draft of the release notes containing
information about substantial changes/updates/bugfixes and their PR number.
Once you have a draft, make sure you solicit feedback from other Ray
developers before publishing. Use the following to get started:
.. code-block:: bash
git pull origin master --tags
git log $(git describe --tags --abbrev=0)..HEAD --pretty=format:"%s" | sort
.. _documentation: https://ray.readthedocs.io/en/latest/installation.html#trying-snapshots-from-master
.. _`documentation for building wheels`: https://github.com/ray-project/ray/blob/master/python/README-building-wheels.md
.. _`ci/stress_tests/run_stress_tests.sh`: https://github.com/ray-project/ray/blob/master/ci/stress_tests/run_stress_tests.sh
.. _`ci/stress_tests/run_application_stress_tests.sh`: https://github.com/ray-project/ray/blob/master/ci/stress_tests/run_application_stress_tests.sh