mirror of
https://github.com/wassname/ray.git
synced 2026-08-18 12:20:14 +08:00
[docs] Add public materials (#6331)
* startup * update tune readme * usingrah
This commit is contained in:
Richard Liaw
@@ -36,3 +36,15 @@ Examples Overview
|
||||
.. customgalleryitem::
|
||||
:tooltip: Distributed Fault-Tolerant BERT training for FAIRSeq using Ray.
|
||||
:description: :doc:`/auto_examples/plot_example-lm`
|
||||
|
||||
.. toctree::
|
||||
:hidden:
|
||||
|
||||
plot_parameter_server.rst
|
||||
plot_example-a3c.rst
|
||||
plot_hyperparameter.rst
|
||||
plot_pong_example.rst
|
||||
plot_lbfgs.rst
|
||||
plot_newsreader.rst
|
||||
plot_streaming.rst
|
||||
plot_example-lm.rst
|
||||
|
||||
+57
-35
@@ -19,21 +19,21 @@ Ray is packaged with the following libraries for accelerating machine learning w
|
||||
- `RLlib`_: Scalable Reinforcement Learning
|
||||
- `Distributed Training <distributed_training.html>`__
|
||||
|
||||
Install Ray with: ``pip install ray``. For nightly wheels, see the `Installation page <installation.html>`__.
|
||||
|
||||
View the `codebase on GitHub`_.
|
||||
|
||||
.. _`codebase on GitHub`: https://github.com/ray-project/ray
|
||||
Star us on `on GitHub`_. You can also get started by visiting our `Tutorials <https://github.com/ray-project/tutorial>`_. For the latest wheels (nightlies), see the `installation page <installation.html>`__.
|
||||
|
||||
.. _`on GitHub`: https://github.com/ray-project/ray
|
||||
|
||||
|
||||
Quick Start
|
||||
-----------
|
||||
|
||||
Execute Python functions in parallel.
|
||||
First, install Ray with: ``pip install ray``
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# Execute Python functions in parallel.
|
||||
|
||||
import ray
|
||||
ray.init()
|
||||
|
||||
@@ -152,15 +152,57 @@ RLlib Quick Start
|
||||
More Information
|
||||
----------------
|
||||
|
||||
- `Tutorial`_
|
||||
- `Blog`_
|
||||
Here are some talks, papers, and press coverage involving Ray and its libraries. Please raise an issue if any of the below links are broken!
|
||||
|
||||
Blog and Press
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
- `Modern Parallel and Distributed Python: A Quick Tutorial on Ray <https://towardsdatascience.com/modern-parallel-and-distributed-python-a-quick-tutorial-on-ray-99f8d70369b8>`_
|
||||
- `Why Every Python Developer Will Love Ray <https://www.datanami.com/2019/11/05/why-every-python-developer-will-love-ray/>`_
|
||||
- `Meet Ray, the Real-Time Machine-Learning Replacement for Spark <https://www.datanami.com/2017/03/28/meet-ray-real-time-machine-learning-replacement-spark/>`_
|
||||
- `Ray: A Distributed System for AI (BAIR) <http://bair.berkeley.edu/blog/2018/01/09/ray/>`_
|
||||
- `10x Faster Parallel Python Without Python Multiprocessing <https://towardsdatascience.com/10x-faster-parallel-python-without-python-multiprocessing-e5017c93cce1>`_
|
||||
- `Implementing A Parameter Server in 15 Lines of Python with Ray <https://ray-project.github.io/2018/07/15/parameter-server-in-fifteen-lines.html>`_
|
||||
- `Ray Distributed AI Framework Curriculum <https://rise.cs.berkeley.edu/blog/ray-intel-curriculum/>`_
|
||||
- `RayOnSpark: Running Emerging AI Applications on Big Data Clusters with Ray and Analytics Zoo <https://medium.com/riselab/rayonspark-running-emerging-ai-applications-on-big-data-clusters-with-ray-and-analytics-zoo-923e0136ed6a>`_
|
||||
- `First user tips for Ray <https://rise.cs.berkeley.edu/blog/ray-tips-for-first-time-users/>`_
|
||||
- [Tune] `Tune: a Python library for fast hyperparameter tuning at any scale <https://towardsdatascience.com/fast-hyperparameter-tuning-at-scale-d428223b081c>`_
|
||||
- [Tune] `Cutting edge hyperparameter tuning with Ray Tune <https://medium.com/riselab/cutting-edge-hyperparameter-tuning-with-ray-tune-be6c0447afdf>`_
|
||||
- [RLlib] `New Library Targets High Speed Reinforcement Learning <https://www.datanami.com/2018/02/01/rays-new-library-targets-high-speed-reinforcement-learning/>`_
|
||||
- [RLlib] `Scaling Multi Agent Reinforcement Learning <http://bair.berkeley.edu/blog/2018/12/12/rllib/>`_
|
||||
- [RLlib] `Functional RL with Keras and Tensorflow Eager <https://bair.berkeley.edu/blog/2019/10/14/functional-rl/>`_
|
||||
- [Modin] `How to Speed up Pandas by 4x with one line of code <https://www.kdnuggets.com/2019/11/speed-up-pandas-4x.html>`_
|
||||
- [Modin] `Quick Tip – Speed up Pandas using Modin <https://pythondata.com/quick-tip-speed-up-pandas-using-modin/>`_
|
||||
- `Ray Blog`_
|
||||
|
||||
.. _`Ray Blog`: https://ray-project.github.io/
|
||||
|
||||
Talks (Videos)
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
- `Programming at any Scale with Ray | SF Python Meetup Sept 2019 <https://www.youtube.com/watch?v=LfpHyIXBhlE>`_
|
||||
- `Ray for Reinforcement Learning | Data Council 2019 <https://www.youtube.com/watch?v=Ayc0ca150HI>`_
|
||||
- `Scaling Interactive Pandas Workflows with Modin <https://www.youtube.com/watch?v=-HjLd_3ahCw>`_
|
||||
- `Ray: A Distributed Execution Framework for AI | SciPy 2018 <https://www.youtube.com/watch?v=D_oz7E4v-U0>`_
|
||||
- `Ray: A Cluster Computing Engine for Reinforcement Learning Applications | Spark Summit <https://www.youtube.com/watch?v=xadZRRB_TeI>`_
|
||||
- `RLlib: Ray Reinforcement Learning Library | RISECamp 2018 <https://www.youtube.com/watch?v=eeRGORQthaQ>`_
|
||||
- `Enabling Composition in Distributed Reinforcement Learning | Spark Summit 2018 <https://www.youtube.com/watch?v=jAEPqjkjth4>`_
|
||||
- `Tune: Distributed Hyperparameter Search | RISECamp 2018 <https://www.youtube.com/watch?v=38Yd_dXW51Q>`_
|
||||
|
||||
Slides
|
||||
~~~~~~
|
||||
- `Talk given at UC Berkeley DS100 <https://docs.google.com/presentation/d/1sF5T_ePR9R6fAi2R6uxehHzXuieme63O2n_5i9m7mVE/edit?usp=sharing>`_
|
||||
- `Talk given in October 2019 <https://docs.google.com/presentation/d/13K0JsogYQX3gUCGhmQ1PQ8HILwEDFysnq0cI2b88XbU/edit?usp=sharing>`_
|
||||
- [Tune] `Talk given at RISECamp 2019 <https://docs.google.com/presentation/d/1v3IldXWrFNMK-vuONlSdEuM82fuGTrNUDuwtfx4axsQ/edit?usp=sharing>`_
|
||||
|
||||
Academic Papers
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
- `Ray paper`_
|
||||
- `Ray HotOS paper`_
|
||||
- `RLlib paper`_
|
||||
- `Tune paper`_
|
||||
|
||||
.. _`Tutorial`: https://github.com/ray-project/tutorial
|
||||
.. _`Blog`: https://ray-project.github.io/
|
||||
.. _`Ray paper`: https://arxiv.org/abs/1712.05889
|
||||
.. _`Ray HotOS paper`: https://arxiv.org/abs/1703.03924
|
||||
.. _`RLlib paper`: https://arxiv.org/abs/1712.09381
|
||||
@@ -190,21 +232,16 @@ Getting Involved
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: -1
|
||||
:caption: Using Ray
|
||||
:caption: Ray Core
|
||||
|
||||
walkthrough.rst
|
||||
actors.rst
|
||||
using-ray-with-gpus.rst
|
||||
serialization.rst
|
||||
memory-management.rst
|
||||
configure.rst
|
||||
troubleshooting.rst
|
||||
advanced.rst
|
||||
using-ray.rst
|
||||
Tutorials <https://github.com/ray-project/tutorial>
|
||||
Examples <auto_examples/overview.rst>
|
||||
package-ref.rst
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: -1
|
||||
:caption: Cluster Setup
|
||||
:caption: Deploying Ray (Cluster Setup)
|
||||
|
||||
autoscaling.rst
|
||||
using-ray-on-a-cluster.rst
|
||||
@@ -212,6 +249,7 @@ Getting Involved
|
||||
deploy-on-kubernetes.rst
|
||||
deploying-on-slurm.rst
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: -1
|
||||
:caption: Tune
|
||||
@@ -255,22 +293,6 @@ Getting Involved
|
||||
async_api.rst
|
||||
serve.rst
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: -1
|
||||
:caption: Examples
|
||||
|
||||
auto_examples/overview.rst
|
||||
auto_examples/plot_parameter_server.rst
|
||||
auto_examples/plot_example-a3c.rst
|
||||
auto_examples/plot_hyperparameter.rst
|
||||
auto_examples/plot_pong_example.rst
|
||||
auto_examples/plot_lbfgs.rst
|
||||
auto_examples/plot_newsreader.rst
|
||||
auto_examples/plot_streaming.rst
|
||||
auto_examples/plot_example-lm.rst
|
||||
using-ray-with-tensorflow.rst
|
||||
using-ray-with-pytorch.rst
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: -1
|
||||
:caption: Development and Internals
|
||||
|
||||
+40
-20
@@ -9,27 +9,38 @@ Tune: A Scalable Hyperparameter Tuning Library
|
||||
|
||||
Tune is a Python library for hyperparameter tuning at any scale. Core features:
|
||||
|
||||
* Launch a multi-node distributed hyperparameter sweep in less than 10 lines of code.
|
||||
* Supports any machine learning framework, including PyTorch, XGBoost, MXNet, and Keras.
|
||||
* Launch a multi-node `distributed hyperparameter sweep <tune-distributed.html>`_ in less than 10 lines of code.
|
||||
* Supports any machine learning framework, including PyTorch, XGBoost, MXNet, and Keras. See `examples here <tune-examples.html>`_.
|
||||
* Natively `integrates with optimization libraries <tune-searchalg.html>`_ such as `HyperOpt <https://github.com/hyperopt/hyperopt>`_, `Bayesian Optimization <https://github.com/fmfn/BayesianOptimization>`_, and `Facebook Ax <http://ax.dev>`_.
|
||||
* Choose among `scalable algorithms <tune-schedulers.html>`_ such as `Population Based Training (PBT)`_, `Vizier's Median Stopping Rule`_, `HyperBand/ASHA`_.
|
||||
* Visualize results with `TensorBoard <https://www.tensorflow.org/get_started/summaries_and_tensorboard>`__.
|
||||
* Choose among scalable SOTA algorithms such as `Population Based Training (PBT)`_, `Vizier's Median Stopping Rule`_, `HyperBand/ASHA`_.
|
||||
* Tune integrates with many optimization libraries such as `Facebook Ax <http://ax.dev>`_, `HyperOpt <https://github.com/hyperopt/hyperopt>`_, and `Bayesian Optimization <https://github.com/fmfn/BayesianOptimization>`_ and enables you to scale them transparently.
|
||||
|
||||
.. _`Population Based Training (PBT)`: tune-schedulers.html#population-based-training-pbt
|
||||
.. _`Vizier's Median Stopping Rule`: tune-schedulers.html#median-stopping-rule
|
||||
.. _`HyperBand/ASHA`: tune-schedulers.html#asynchronous-hyperband
|
||||
|
||||
For more information, check out:
|
||||
|
||||
* `Code <https://github.com/ray-project/ray/tree/master/python/ray/tune>`__: GitHub repository for Tune.
|
||||
* `User Guide <tune-usage.html>`__: A comprehensive overview on how to use Tune's features.
|
||||
* `Tutorial Notebooks <https://github.com/ray-project/tutorial/blob/master/tune_exercises/>`__: Our tutorial notebooks of using Tune with Keras or PyTorch.
|
||||
|
||||
**Try out a tutorial notebook on Colab**:
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<a href="https://colab.research.google.com/github/ray-project/tutorial/blob/master/tune_exercises/exercise_1_basics.ipynb" target="_parent">
|
||||
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Tune Tutorial"/>
|
||||
</a>
|
||||
|
||||
Quick Start
|
||||
-----------
|
||||
|
||||
.. note::
|
||||
To run this example, you will need to install the following:
|
||||
|
||||
To run this example, you will need to install the following:
|
||||
.. code-block:: bash
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install ray[tune] torch torchvision filelock
|
||||
$ pip install ray[tune] torch torchvision filelock
|
||||
|
||||
|
||||
This example runs a small grid search to train a CNN using PyTorch and Tune.
|
||||
@@ -47,10 +58,14 @@ If TensorBoard is installed, automatically visualize all trial results:
|
||||
|
||||
|
||||
.. image:: images/tune-start-tb.png
|
||||
:scale: 30%
|
||||
:align: center
|
||||
|
||||
If using TF2 and TensorBoard, Tune will also automatically generate TensorBoard HParams output:
|
||||
|
||||
.. image:: images/tune-hparams-coord.png
|
||||
:scale: 20%
|
||||
:align: center
|
||||
|
||||
Distributed Quick Start
|
||||
-----------------------
|
||||
@@ -100,18 +115,27 @@ Take a look at the `Distributed Experiments <tune-distributed.html>`_ documentat
|
||||
2. Using AWS and GCP
|
||||
3. Spot instance usage/pre-emptible instances, and more.
|
||||
|
||||
Getting Started
|
||||
Talks and Blogs
|
||||
---------------
|
||||
|
||||
* `Code <https://github.com/ray-project/ray/tree/master/python/ray/tune>`__: GitHub repository for Tune.
|
||||
* `User Guide <tune-usage.html>`__: A comprehensive overview on how to use Tune's features.
|
||||
* `Tutorial Notebook <https://github.com/ray-project/tutorial/blob/master/tune_exercises/>`__: Our tutorial notebooks of using Tune with Keras or PyTorch.
|
||||
Below are some blog posts and talks about Tune:
|
||||
|
||||
Contribute to Tune
|
||||
------------------
|
||||
- [blog] `Tune: a Python library for fast hyperparameter tuning at any scale <https://towardsdatascience.com/fast-hyperparameter-tuning-at-scale-d428223b081c>`_
|
||||
- [blog] `Cutting edge hyperparameter tuning with Ray Tune <https://medium.com/riselab/cutting-edge-hyperparameter-tuning-with-ray-tune-be6c0447afdf>`_
|
||||
- [blog] `Simple hyperparameter and architecture search in tensorflow with Ray Tune <http://louiskirsch.com/ai/ray-tune>`_
|
||||
- [slides] `Talk given at RISECamp 2019 <https://docs.google.com/presentation/d/1v3IldXWrFNMK-vuONlSdEuM82fuGTrNUDuwtfx4axsQ/edit?usp=sharing>`_
|
||||
- [Talk] `Talk given at RISECamp 2018 <https://www.youtube.com/watch?v=38Yd_dXW51Q>`_
|
||||
|
||||
Take a look at our `Contributor Guide <tune-contrib.html>`__ for guidelines on contributing.
|
||||
Open Source Projects using Tune
|
||||
-------------------------------
|
||||
|
||||
Here are some of the popular open source repositories and research projects that leverage Tune. Feel free to submit a pull-request adding (or requesting a removal!) of a listed project.
|
||||
|
||||
- `Softlearning <https://github.com/rail-berkeley/softlearning>`_: Softlearning is a reinforcement learning framework for training maximum entropy policies in continuous domains. Includes the official implementation of the Soft Actor-Critic algorithm.
|
||||
- `Flambe <https://github.com/asappresearch/flambe>`_: An ML framework to accelerate research and its path to production. See `flambe.ai <flambe.ai>`_.
|
||||
- `Population Based Augmentation <https://github.com/arcelien/pba>`_: Population Based Augmentation (PBA) is a algorithm that quickly and efficiently learns data augmentation functions for neural network training. PBA matches state-of-the-art results on CIFAR with one thousand times less compute.
|
||||
- `Fast AutoAugment by Kakao <https://github.com/kakaobrain/fast-autoaugment>`_: Fast AutoAugment (Accepted at NeurIPS 2019) learns augmentation policies using a more efficient search strategy based on density matching.
|
||||
- `Allentune <https://github.com/allenai/allentune>`_: Hyperparameter Search for AllenNLP from AllenAI.
|
||||
|
||||
Citing Tune
|
||||
-----------
|
||||
@@ -127,7 +151,3 @@ If Tune helps you in your academic research, you are encouraged to cite `our pap
|
||||
journal={arXiv preprint arXiv:1807.05118},
|
||||
year={2018}
|
||||
}
|
||||
|
||||
|
||||
.. _HyperOpt with HyperBand: https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperopt_example.py
|
||||
.. _Nevergrad with HyperBand: https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/nevergrad_example.py
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
Using Ray
|
||||
=========
|
||||
|
||||
If you’re brand new to Ray, we recommend starting with our `tutorials <https://github.com/ray-project/tutorial>`_.
|
||||
|
||||
Below, you'll find information ranging from beginner material (like our `walkthrough <walkthrough.html>`_) to `advanced usage <advanced.html>`_. There are also detailed instructions on how to work with Ray concepts such as Actors and managing GPUs.
|
||||
|
||||
Finally, we've also included some content on using core Ray APIs with `Tensorflow <using-ray-with-tensorflow.html>`_ and `PyTorch <using-ray-with-pytorch.html>`_.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: -1
|
||||
|
||||
walkthrough.rst
|
||||
actors.rst
|
||||
using-ray-with-gpus.rst
|
||||
serialization.rst
|
||||
memory-management.rst
|
||||
configure.rst
|
||||
troubleshooting.rst
|
||||
advanced.rst
|
||||
using-ray-with-tensorflow.rst
|
||||
using-ray-with-pytorch.rst
|
||||
Reference in New Issue
Block a user