wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-05 13:21:03 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Testing] Multi-node Training+Tune Long Running Test (#8966)

Browse Source
This commit is contained in:
Amog Kamsetty
2020-06-22 14:49:16 -07:00
committed by GitHub
parent 7a110b9401
commit f95ab4f506
9 changed files with 359 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/dev/RELEASE_PROCESS.rst
+13 -3
View File
@@ -26,7 +26,7 @@ This document describes the process for creating new releases.
4. **Testing:** Before releasing, the following sets of tests should be run.
The results of each of these tests for previous releases are checked in
under ``doc/dev/release_tests``, and should be compared against to identify
under ``doc/dev/release_logs``, and should be compared against to identify
any regressions.
1. Long-running tests
@@ -35,10 +35,20 @@ This document describes the process for creating new releases.
ray/ci/long_running_tests/README.rst
Follow the instructions to kick off the tests and check the status of the workloads
These tests should run for at least 24 hours (printing new iterations and CPU load
Follow the instructions to kick off the tests and check the status of the workloads.
These tests should run for at least 24 hours without erroring or hanging (ensure that it is printing new iterations and CPU load is
stable in the AWS console).
2. Long-running multi-node tests
.. code-block:: bash
ray/ci/long_running_distributed_tests/README.rst
Follow the instructions to kick off the tests and check the status of the workloads.
These suite of tests are similar to the standard long running tests, except these actually run in a multi-node cluster instead of just a simulated one.
These tests should also run for at least 24 hours without erroring or hanging.
2. Multi-node regression tests
Follow the same instruction as long running stress tests. The large scale distributed