diff --git a/README.rst b/README.rst index 74b447c12..7879e8b74 100644 --- a/README.rst +++ b/README.rst @@ -15,10 +15,19 @@ Ray is packaged with the following libraries for accelerating machine learning w - `RaySGD `__: Distributed Training Wrappers - `Ray Serve`_: Scalable and Programmable Serving +There are also many `community integrations `_ with Ray, including `Dask`_, `MARS`_, `Modin`_, `Horovod`_, `Hugging Face`_, `Scikit-learn`_, and others. Check out the `full list of Ray distributed libraries here `_. + Install Ray with: ``pip install ray``. For nightly wheels, see the `Installation page `__. -**NOTE:** As of Ray 0.8.1, Python 2 is no longer supported. +.. _`Modin`: https://github.com/modin-project/modin +.. _`Hugging Face`: https://huggingface.co/transformers/main_classes/trainer.html#transformers.Trainer.hyperparameter_search +.. _`MARS`: mars-on-ray.html +.. _`Dask`: dask-on-ray.html +.. _`Horovod`: https://horovod.readthedocs.io/en/stable/ray_include.html +.. _`Scikit-learn`: joblib.html + + Quick Start ----------- @@ -254,7 +263,6 @@ This example runs serves a scikit-learn gradient boosting classifier. .. _`Ray Serve`: https://docs.ray.io/en/latest/serve/index.html - More Information ---------------- diff --git a/doc/requirements-rtd.txt b/doc/requirements-rtd.txt index 1c6cd5322..2afa6722c 100644 --- a/doc/requirements-rtd.txt +++ b/doc/requirements-rtd.txt @@ -8,3 +8,4 @@ commonmark==0.8.1 recommonmark==0.5.0 sphinx<2 readthedocs-sphinx-ext<1.1 +sphinx-book-theme diff --git a/doc/source/images/classyvision.png b/doc/source/images/classyvision.png new file mode 100644 index 000000000..3b7fa26d1 Binary files /dev/null and b/doc/source/images/classyvision.png differ diff --git a/doc/source/images/dask.png b/doc/source/images/dask.png new file mode 100644 index 000000000..573a43a56 Binary files /dev/null and b/doc/source/images/dask.png differ diff --git a/doc/source/images/flambe.png b/doc/source/images/flambe.png new file mode 100644 index 000000000..4780b2b08 Binary files /dev/null and b/doc/source/images/flambe.png differ diff --git a/doc/source/images/horovod.png b/doc/source/images/horovod.png new file mode 100644 index 000000000..c541d1d86 Binary files /dev/null and b/doc/source/images/horovod.png differ diff --git a/doc/source/images/hugging.png b/doc/source/images/hugging.png new file mode 100644 index 000000000..c50928eea Binary files /dev/null and b/doc/source/images/hugging.png differ diff --git a/doc/source/images/mars.png b/doc/source/images/mars.png new file mode 100644 index 000000000..f454f99a8 Binary files /dev/null and b/doc/source/images/mars.png differ diff --git a/doc/source/images/modin.png b/doc/source/images/modin.png new file mode 100644 index 000000000..ef5970a09 Binary files /dev/null and b/doc/source/images/modin.png differ diff --git a/doc/source/images/pycaret.png b/doc/source/images/pycaret.png new file mode 100644 index 000000000..bac4c4185 Binary files /dev/null and b/doc/source/images/pycaret.png differ diff --git a/doc/source/images/seldon.png b/doc/source/images/seldon.png new file mode 100644 index 000000000..941c21131 Binary files /dev/null and b/doc/source/images/seldon.png differ diff --git a/doc/source/images/spacy.png b/doc/source/images/spacy.png new file mode 100644 index 000000000..d0a6e6a5b Binary files /dev/null and b/doc/source/images/spacy.png differ diff --git a/doc/source/images/zoo.png b/doc/source/images/zoo.png new file mode 100644 index 000000000..f7fe925fb Binary files /dev/null and b/doc/source/images/zoo.png differ diff --git a/doc/source/index.rst b/doc/source/index.rst index e306a5244..8aa8ebe4d 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -195,7 +195,6 @@ Papers ray-overview/index.rst installation.rst - oss-users.rst .. toctree:: :hidden: @@ -277,14 +276,14 @@ Papers .. toctree:: :hidden: :maxdepth: -1 - :caption: Community Libraries + :caption: More Libraries multiprocessing.rst joblib.rst iter.rst - pandas_on_ray.rst dask-on-ray.rst mars-on-ray.rst + oss-users.rst .. toctree:: :hidden: diff --git a/doc/source/multiprocessing.rst b/doc/source/multiprocessing.rst index 9db205df6..cce9b224b 100644 --- a/doc/source/multiprocessing.rst +++ b/doc/source/multiprocessing.rst @@ -1,3 +1,5 @@ +.. _ray-multiprocessing: + Distributed multiprocessing.Pool ================================ diff --git a/doc/source/oss-users.rst b/doc/source/oss-users.rst index 42089fdac..ac592de09 100644 --- a/doc/source/oss-users.rst +++ b/doc/source/oss-users.rst @@ -1,79 +1,128 @@ .. _ray-oss-list: -Community libraries that integrate with Ray -=========================================== +Community Integrations +====================== -This page lists libraries that integrate with Ray for distributed execution. If you'd like to add your project to this list, feel free to file a pull request or open an issue on GitHub. +This page lists libraries that have integrations with Ray for distributed execution. If you'd like to add your project to this list, feel free to file a pull request or open an issue on GitHub. + +ClassyVision |classyvision| +--------------------------- + +Classy Vision is a new end-to-end, PyTorch-based framework for large-scale training of state-of-the-art image and video classification models. The library features a modular, flexible design that allows anyone to train machine learning models on top of PyTorch using very simple abstractions. -Horovod -------- +[`Link to integration `__] + +Dask |dask| +----------- + +Dask provides advanced parallelism for analytics, enabling performance at scale for the tools you love. Dask uses existing Python APIs and data structures to make it easy to switch between Numpy, Pandas, Scikit-learn to their Dask-powered equivalents. + +[`Link to integration `__] + +Flambe |flambe| +--------------- + +Flambé is a machine learning experimentation framework built to accelerate the entire research life cycle. Flambé’s main objective is to provide a unified interface for prototyping models, running experiments containing complex pipelines, monitoring those experiments in real-time, reporting results, and deploying a final model for inference. + +Github: `https://github.com/asappresearch/flambe `_ + +MARS |mars| +----------- + +Mars is a tensor-based unified framework for large-scale data computation which scales Numpy, Pandas and Scikit-learn. Mars can scale in to a single machine, and scale out to a cluster with thousands of machines. + + +[`Link to integration `__] + +Modin |modin| +------------- + +Scale your pandas workflows by changing one line of code. Modin transparently distributes the data and computation so that all you need to do is continue using the pandas API as you were before installing Modin. + +GitHub: `https://github.com/modin-project/modin `_ + +Horovod |horovod| +----------------- Horovod is a distributed deep learning training framework for TensorFlow, Keras, PyTorch, and Apache MXNet. The goal of Horovod is to make distributed deep learning fast and easy to use. [`Link to integration `__] -MARS ----- - -Mars is a tensor-based unified framework for large-scale data computation which scales Numpy, Pandas and Scikit-learn. - -[:ref:`Link to integration `] - -Spacy ------ -spaCy is a library for advanced Natural Language Processing in Python and Cython. It's built on the very latest research, and was designed from day one to be used in real products. - -[`Link to integration `__] - - -Hugging Face Transformers -------------------------- +Hugging Face Transformers |hugging| +----------------------------------- State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0. [`Link to integration `__] -Seldon Alibi ------------- - -Alibi is an open source Python library aimed at machine learning model inspection and interpretation. The focus of the library is to provide high-quality implementations of black-box, white-box, local and global explanation methods for classification and regression models. - -Github: `https://github.com/SeldonIO/alibi `__ - - -Intel Analytics Zoo -------------------- +Intel Analytics Zoo |zoo| +------------------------- Analytics Zoo seamless scales TensorFlow, Keras and PyTorch to distributed big data (using Spark, Flink & Ray). [`Link to integration `__] -Modin ------ +PyCaret |pycaret| +----------------- -Scale your pandas workflows by changing one line of code. - -Github: `https://github.com/modin-project/modin/ `__ - - -ClassyVision ------------- - -An end-to-end framework for image and video classification. - -[`Link to integration `__] - -PyCaret -------- - -An open-source, low-code machine learning library in Python. +PyCaret is an open source low-code machine learning library in Python that aims to reduce the hypothesis to insights cycle time in a ML experiment. It enables data scientists to perform end-to-end experiments quickly and efficiently. Github: `https://github.com/pycaret/pycaret/ `_ -Flambe ------- +Seldon Alibi |seldon| +--------------------- -Flambé is a machine learning experimentation framework built to accelerate the entire research life cycle. Flambé’s main objective is to provide a unified interface for prototyping models, running experiments containing complex pipelines, monitoring those experiments in real-time, reporting results, and deploying a final model for inference. +Alibi is an open source Python library aimed at machine learning model inspection and interpretation. The focus of the library is to provide high-quality implementations of black-box, white-box, local and global explanation methods for classification and regression models. -Github: `https://github.com/asappresearch/flambe `_ +Github: `https://github.com/SeldonIO/alibi `__ + +Spacy |spacy| +------------- +spaCy is a library for advanced Natural Language Processing in Python and Cython. It's built on the very latest research, and was designed from day one to be used in real products. + +[`Link to integration `__] + +.. |classyvision| image:: images/classyvision.png + :class: inline-figure + :height: 30 + +.. |dask| image:: images/dask.png + :class: inline-figure + :height: 30 + +.. |flambe| image:: images/flambe.png + :class: inline-figure + :height: 30 + +.. |mars| image:: images/mars.png + :class: inline-figure + :height: 30 + +.. |modin| image:: images/modin.png + :class: inline-figure + :height: 30 + +.. |horovod| image:: images/horovod.png + :class: inline-figure + :height: 30 + +.. |hugging| image:: images/hugging.png + :class: inline-figure + :height: 30 + +.. |zoo| image:: images/zoo.png + :class: inline-figure + :height: 30 + +.. |pycaret| image:: images/pycaret.png + :class: inline-figure + :height: 30 + +.. |seldon| image:: images/seldon.png + :class: inline-figure + :height: 30 + +.. |spacy| image:: images/spacy.png + :class: inline-figure + :height: 30 diff --git a/doc/source/pandas_on_ray.rst b/doc/source/pandas_on_ray.rst deleted file mode 100644 index 4378d12e4..000000000 --- a/doc/source/pandas_on_ray.rst +++ /dev/null @@ -1,9 +0,0 @@ -Pandas on Ray -============= - -**Pandas on Ray has moved to Modin!** - -Pandas on Ray has moved into the `Modin project`_ with the intention of -unifying the DataFrame APIs. - -.. _`Modin project`: https://github.com/modin-project/modin diff --git a/doc/source/ray-overview/basics.rst b/doc/source/ray-overview/basics.rst index 1b672b42c..1f2410681 100644 --- a/doc/source/ray-overview/basics.rst +++ b/doc/source/ray-overview/basics.rst @@ -18,10 +18,11 @@ On top of **Ray Core** are several libraries for solving problems in machine lea - :ref:`sgd-index` - :ref:`rayserve` -There are also many :ref:`community libraries ` that integrate with Ray, including `Modin`_, `Hugging Face Transformers`_, :doc:`../dask-on-ray`, `MARS`_, `Horovod`_, and others. -Check out the :ref:`full list of Ray distributed libraries here `. +There are also many :ref:`community integrations ` with Ray, including `Dask`_, `MARS`_, `Modin`_, `Horovod`_, `Hugging Face`_, `Scikit-learn`_, and others. Check out the :ref:`full list of Ray distributed libraries here `. .. _`Modin`: https://github.com/modin-project/modin -.. _`Hugging Face Transformers`: https://huggingface.co/transformers/main_classes/trainer.html#transformers.Trainer.hyperparameter_search +.. _`Hugging Face`: https://huggingface.co/transformers/main_classes/trainer.html#transformers.Trainer.hyperparameter_search .. _`MARS`: mars-on-ray.html +.. _`Dask`: dask-on-ray.html .. _`Horovod`: https://horovod.readthedocs.io/en/stable/ray_include.html +.. _`Scikit-learn`: joblib.html