diff --git a/doc/examples/overview.rst b/doc/examples/overview.rst index 3fe6d18ad..9ede56417 100644 --- a/doc/examples/overview.rst +++ b/doc/examples/overview.rst @@ -36,3 +36,15 @@ Examples Overview .. customgalleryitem:: :tooltip: Distributed Fault-Tolerant BERT training for FAIRSeq using Ray. :description: :doc:`/auto_examples/plot_example-lm` + +.. toctree:: + :hidden: + + plot_parameter_server.rst + plot_example-a3c.rst + plot_hyperparameter.rst + plot_pong_example.rst + plot_lbfgs.rst + plot_newsreader.rst + plot_streaming.rst + plot_example-lm.rst diff --git a/doc/source/index.rst b/doc/source/index.rst index 3305eae28..55d172674 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -19,21 +19,21 @@ Ray is packaged with the following libraries for accelerating machine learning w - `RLlib`_: Scalable Reinforcement Learning - `Distributed Training `__ -Install Ray with: ``pip install ray``. For nightly wheels, see the `Installation page `__. -View the `codebase on GitHub`_. - -.. _`codebase on GitHub`: https://github.com/ray-project/ray +Star us on `on GitHub`_. You can also get started by visiting our `Tutorials `_. For the latest wheels (nightlies), see the `installation page `__. +.. _`on GitHub`: https://github.com/ray-project/ray Quick Start ----------- -Execute Python functions in parallel. +First, install Ray with: ``pip install ray`` .. code-block:: python + # Execute Python functions in parallel. + import ray ray.init() @@ -152,15 +152,57 @@ RLlib Quick Start More Information ---------------- -- `Tutorial`_ -- `Blog`_ +Here are some talks, papers, and press coverage involving Ray and its libraries. Please raise an issue if any of the below links are broken! + +Blog and Press +~~~~~~~~~~~~~~ + + - `Modern Parallel and Distributed Python: A Quick Tutorial on Ray `_ + - `Why Every Python Developer Will Love Ray `_ + - `Meet Ray, the Real-Time Machine-Learning Replacement for Spark `_ + - `Ray: A Distributed System for AI (BAIR) `_ + - `10x Faster Parallel Python Without Python Multiprocessing `_ + - `Implementing A Parameter Server in 15 Lines of Python with Ray `_ + - `Ray Distributed AI Framework Curriculum `_ + - `RayOnSpark: Running Emerging AI Applications on Big Data Clusters with Ray and Analytics Zoo `_ + - `First user tips for Ray `_ + - [Tune] `Tune: a Python library for fast hyperparameter tuning at any scale `_ + - [Tune] `Cutting edge hyperparameter tuning with Ray Tune `_ + - [RLlib] `New Library Targets High Speed Reinforcement Learning `_ + - [RLlib] `Scaling Multi Agent Reinforcement Learning `_ + - [RLlib] `Functional RL with Keras and Tensorflow Eager `_ + - [Modin] `How to Speed up Pandas by 4x with one line of code `_ + - [Modin] `Quick Tip – Speed up Pandas using Modin `_ + - `Ray Blog`_ + +.. _`Ray Blog`: https://ray-project.github.io/ + +Talks (Videos) +~~~~~~~~~~~~~~ + + - `Programming at any Scale with Ray | SF Python Meetup Sept 2019 `_ + - `Ray for Reinforcement Learning | Data Council 2019 `_ + - `Scaling Interactive Pandas Workflows with Modin `_ + - `Ray: A Distributed Execution Framework for AI | SciPy 2018 `_ + - `Ray: A Cluster Computing Engine for Reinforcement Learning Applications | Spark Summit `_ + - `RLlib: Ray Reinforcement Learning Library | RISECamp 2018 `_ + - `Enabling Composition in Distributed Reinforcement Learning | Spark Summit 2018 `_ + - `Tune: Distributed Hyperparameter Search | RISECamp 2018 `_ + +Slides +~~~~~~ +- `Talk given at UC Berkeley DS100 `_ +- `Talk given in October 2019 `_ +- [Tune] `Talk given at RISECamp 2019 `_ + +Academic Papers +~~~~~~~~~~~~~~~ + - `Ray paper`_ - `Ray HotOS paper`_ - `RLlib paper`_ - `Tune paper`_ -.. _`Tutorial`: https://github.com/ray-project/tutorial -.. _`Blog`: https://ray-project.github.io/ .. _`Ray paper`: https://arxiv.org/abs/1712.05889 .. _`Ray HotOS paper`: https://arxiv.org/abs/1703.03924 .. _`RLlib paper`: https://arxiv.org/abs/1712.09381 @@ -190,21 +232,16 @@ Getting Involved .. toctree:: :maxdepth: -1 - :caption: Using Ray + :caption: Ray Core - walkthrough.rst - actors.rst - using-ray-with-gpus.rst - serialization.rst - memory-management.rst - configure.rst - troubleshooting.rst - advanced.rst + using-ray.rst + Tutorials + Examples package-ref.rst .. toctree:: :maxdepth: -1 - :caption: Cluster Setup + :caption: Deploying Ray (Cluster Setup) autoscaling.rst using-ray-on-a-cluster.rst @@ -212,6 +249,7 @@ Getting Involved deploy-on-kubernetes.rst deploying-on-slurm.rst + .. toctree:: :maxdepth: -1 :caption: Tune @@ -255,22 +293,6 @@ Getting Involved async_api.rst serve.rst -.. toctree:: - :maxdepth: -1 - :caption: Examples - - auto_examples/overview.rst - auto_examples/plot_parameter_server.rst - auto_examples/plot_example-a3c.rst - auto_examples/plot_hyperparameter.rst - auto_examples/plot_pong_example.rst - auto_examples/plot_lbfgs.rst - auto_examples/plot_newsreader.rst - auto_examples/plot_streaming.rst - auto_examples/plot_example-lm.rst - using-ray-with-tensorflow.rst - using-ray-with-pytorch.rst - .. toctree:: :maxdepth: -1 :caption: Development and Internals diff --git a/doc/source/tune.rst b/doc/source/tune.rst index 4036a0c0a..85da4179e 100644 --- a/doc/source/tune.rst +++ b/doc/source/tune.rst @@ -9,27 +9,38 @@ Tune: A Scalable Hyperparameter Tuning Library Tune is a Python library for hyperparameter tuning at any scale. Core features: - * Launch a multi-node distributed hyperparameter sweep in less than 10 lines of code. - * Supports any machine learning framework, including PyTorch, XGBoost, MXNet, and Keras. + * Launch a multi-node `distributed hyperparameter sweep `_ in less than 10 lines of code. + * Supports any machine learning framework, including PyTorch, XGBoost, MXNet, and Keras. See `examples here `_. + * Natively `integrates with optimization libraries `_ such as `HyperOpt `_, `Bayesian Optimization `_, and `Facebook Ax `_. + * Choose among `scalable algorithms `_ such as `Population Based Training (PBT)`_, `Vizier's Median Stopping Rule`_, `HyperBand/ASHA`_. * Visualize results with `TensorBoard `__. - * Choose among scalable SOTA algorithms such as `Population Based Training (PBT)`_, `Vizier's Median Stopping Rule`_, `HyperBand/ASHA`_. - * Tune integrates with many optimization libraries such as `Facebook Ax `_, `HyperOpt `_, and `Bayesian Optimization `_ and enables you to scale them transparently. .. _`Population Based Training (PBT)`: tune-schedulers.html#population-based-training-pbt .. _`Vizier's Median Stopping Rule`: tune-schedulers.html#median-stopping-rule .. _`HyperBand/ASHA`: tune-schedulers.html#asynchronous-hyperband +For more information, check out: + + * `Code `__: GitHub repository for Tune. + * `User Guide `__: A comprehensive overview on how to use Tune's features. + * `Tutorial Notebooks `__: Our tutorial notebooks of using Tune with Keras or PyTorch. + +**Try out a tutorial notebook on Colab**: + +.. raw:: html + + + Tune Tutorial + Quick Start ----------- -.. note:: +To run this example, you will need to install the following: - To run this example, you will need to install the following: +.. code-block:: bash - .. code-block:: bash - - $ pip install ray[tune] torch torchvision filelock + $ pip install ray[tune] torch torchvision filelock This example runs a small grid search to train a CNN using PyTorch and Tune. @@ -47,10 +58,14 @@ If TensorBoard is installed, automatically visualize all trial results: .. image:: images/tune-start-tb.png + :scale: 30% + :align: center If using TF2 and TensorBoard, Tune will also automatically generate TensorBoard HParams output: .. image:: images/tune-hparams-coord.png + :scale: 20% + :align: center Distributed Quick Start ----------------------- @@ -100,18 +115,27 @@ Take a look at the `Distributed Experiments `_ documentat 2. Using AWS and GCP 3. Spot instance usage/pre-emptible instances, and more. -Getting Started +Talks and Blogs --------------- - * `Code `__: GitHub repository for Tune. - * `User Guide `__: A comprehensive overview on how to use Tune's features. - * `Tutorial Notebook `__: Our tutorial notebooks of using Tune with Keras or PyTorch. +Below are some blog posts and talks about Tune: -Contribute to Tune ------------------- + - [blog] `Tune: a Python library for fast hyperparameter tuning at any scale `_ + - [blog] `Cutting edge hyperparameter tuning with Ray Tune `_ + - [blog] `Simple hyperparameter and architecture search in tensorflow with Ray Tune `_ + - [slides] `Talk given at RISECamp 2019 `_ + - [Talk] `Talk given at RISECamp 2018 `_ -Take a look at our `Contributor Guide `__ for guidelines on contributing. +Open Source Projects using Tune +------------------------------- +Here are some of the popular open source repositories and research projects that leverage Tune. Feel free to submit a pull-request adding (or requesting a removal!) of a listed project. + + - `Softlearning `_: Softlearning is a reinforcement learning framework for training maximum entropy policies in continuous domains. Includes the official implementation of the Soft Actor-Critic algorithm. + - `Flambe `_: An ML framework to accelerate research and its path to production. See `flambe.ai `_. + - `Population Based Augmentation `_: Population Based Augmentation (PBA) is a algorithm that quickly and efficiently learns data augmentation functions for neural network training. PBA matches state-of-the-art results on CIFAR with one thousand times less compute. + - `Fast AutoAugment by Kakao `_: Fast AutoAugment (Accepted at NeurIPS 2019) learns augmentation policies using a more efficient search strategy based on density matching. + - `Allentune `_: Hyperparameter Search for AllenNLP from AllenAI. Citing Tune ----------- @@ -127,7 +151,3 @@ If Tune helps you in your academic research, you are encouraged to cite `our pap journal={arXiv preprint arXiv:1807.05118}, year={2018} } - - -.. _HyperOpt with HyperBand: https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperopt_example.py -.. _Nevergrad with HyperBand: https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/nevergrad_example.py diff --git a/doc/source/using-ray.rst b/doc/source/using-ray.rst new file mode 100644 index 000000000..d4128e620 --- /dev/null +++ b/doc/source/using-ray.rst @@ -0,0 +1,22 @@ +Using Ray +========= + +If you’re brand new to Ray, we recommend starting with our `tutorials `_. + +Below, you'll find information ranging from beginner material (like our `walkthrough `_) to `advanced usage `_. There are also detailed instructions on how to work with Ray concepts such as Actors and managing GPUs. + +Finally, we've also included some content on using core Ray APIs with `Tensorflow `_ and `PyTorch `_. + +.. toctree:: + :maxdepth: -1 + + walkthrough.rst + actors.rst + using-ray-with-gpus.rst + serialization.rst + memory-management.rst + configure.rst + troubleshooting.rst + advanced.rst + using-ray-with-tensorflow.rst + using-ray-with-pytorch.rst