mirror of
https://github.com/wassname/ray.git
synced 2026-09-12 12:51:15 +08:00
[Serve] RayServe TF, PyTorch, Sklearn Examples (#8156)
This commit is contained in:
Simon Mo
@@ -285,6 +285,9 @@ Getting Involved
|
||||
:caption: RayServe
|
||||
|
||||
rayserve/overview.rst
|
||||
rayserve/tutorials/tensorflow-tutorial.rst
|
||||
rayserve/tutorials/pytorch-tutorial.rst
|
||||
rayserve/tutorials/sklearn-tutorial.rst
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: -1
|
||||
|
||||
@@ -8,6 +8,8 @@ RayServe: Scalable and Programmable Serving
|
||||
:height: 250px
|
||||
:width: 400px
|
||||
|
||||
.. _rayserve-overview:
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
@@ -92,6 +94,7 @@ To follow along, you'll need to make the necessary imports.
|
||||
from ray import serve
|
||||
serve.init() # initializes serve and Ray
|
||||
|
||||
.. _serve-endpoint:
|
||||
|
||||
Endpoints
|
||||
~~~~~~~~~
|
||||
@@ -105,6 +108,8 @@ model that you'll be serving. To create one, we'll simply specify the name, rout
|
||||
|
||||
serve.create_endpoint("simple_endpoint", "/simple")
|
||||
|
||||
.. _serve-backend:
|
||||
|
||||
Backends
|
||||
~~~~~~~~
|
||||
|
||||
@@ -209,6 +214,15 @@ You can also have RayServe batch requests for performance. You'll configure this
|
||||
serve.link("counter1", "counter1")
|
||||
|
||||
Other Resources
|
||||
----------------
|
||||
---------------
|
||||
|
||||
More coming soon!
|
||||
.. _serve_frameworks:
|
||||
|
||||
Frameworks
|
||||
~~~~~~~~~~
|
||||
RayServe makes it easy to deploy models from all popular frameworks.
|
||||
Learn more about how to deploy your model in the following tutorials:
|
||||
|
||||
- :ref:`Tensorflow & Keras <serve-tensorflow-tutorial>`
|
||||
- :ref:`PyTorch <serve-pytorch-tutorial>`
|
||||
- :ref:`Scikit-Learn <serve-sklearn-tutorial>`
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
.. _serve-pytorch-tutorial:
|
||||
|
||||
PyTorch Tutorial
|
||||
================
|
||||
|
||||
In this guide, we will load and serve a PyTorch Resnet Model.
|
||||
In particular, we show:
|
||||
|
||||
- How to load the model from PyTorch's pre-trained modelzoo.
|
||||
- How to parse the JSON request, transform the payload and evaluated in the model.
|
||||
|
||||
Please see the :ref:`overview <rayserve-overview>` to learn more general information about RayServe.
|
||||
|
||||
This tutorial requires Pytorch and Torchvision installed in your system. RayServe
|
||||
is :ref:`framework agnostic <serve_frameworks>` and work with any version of PyTorch.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install torch torchvision
|
||||
|
||||
Let's import RayServe and some other helpers.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_pytorch.py
|
||||
:start-after: __doc_import_begin__
|
||||
:end-before: __doc_import_end__
|
||||
|
||||
|
||||
Services are just defined as normal classes with ``__init__`` and ``__call__`` methods.
|
||||
The ``__call__`` method will be invoked per request.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_pytorch.py
|
||||
:start-after: __doc_define_servable_begin__
|
||||
:end-before: __doc_define_servable_end__
|
||||
|
||||
Now that we've defined our services, let's deploy the model to RayServe. We will
|
||||
define an :ref:`endpoint <serve-endpoint>` for the route representing the digit classifier task, a
|
||||
:ref:`backend <serve-backend>` correspond the physical implementation, and connect them together.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_pytorch.py
|
||||
:start-after: __doc_deploy_begin__
|
||||
:end-before: __doc_deploy_end__
|
||||
|
||||
Let's query it!
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_pytorch.py
|
||||
:start-after: __doc_query_begin__
|
||||
:end-before: __doc_query_end__
|
||||
@@ -0,0 +1,51 @@
|
||||
.. _serve-sklearn-tutorial:
|
||||
|
||||
Scikit-Learn Tutorial
|
||||
=====================
|
||||
|
||||
In this guide, we will train and deploy a simple Scikit-Learn classifier.
|
||||
In particular, we show:
|
||||
|
||||
- How to load the model from file system in your RayServe definition
|
||||
- How to parse the JSON request and evaluated in sklearn model
|
||||
|
||||
Please see the :ref:`overview <rayserve-overview>` to learn more general information about RayServe.
|
||||
|
||||
RayServe supports :ref:`arbitrary frameworks <serve_frameworks>`. You can use any version of sklearn.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install scikit-learn
|
||||
|
||||
Let's import RayServe and some other helpers.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_sklearn.py
|
||||
:start-after: __doc_import_begin__
|
||||
:end-before: __doc_import_end__
|
||||
|
||||
We will train a logistic regression with the iris dataset.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_sklearn.py
|
||||
:start-after: __doc_train_model_begin__
|
||||
:end-before: __doc_train_model_end__
|
||||
|
||||
Services are just defined as normal classes with ``__init__`` and ``__call__`` methods.
|
||||
The ``__call__`` method will be invoked per request.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_sklearn.py
|
||||
:start-after: __doc_define_servable_begin__
|
||||
:end-before: __doc_define_servable_end__
|
||||
|
||||
Now that we've defined our services, let's deploy the model to RayServe. We will
|
||||
define an :ref:`endpoint <serve-endpoint>` for the route representing the classifier task, a
|
||||
:ref:`backend <serve-backend>` correspond the physical implementation, and connect them together.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_sklearn.py
|
||||
:start-after: __doc_deploy_begin__
|
||||
:end-before: __doc_deploy_end__
|
||||
|
||||
Let's query it!
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_sklearn.py
|
||||
:start-after: __doc_query_begin__
|
||||
:end-before: __doc_query_end__
|
||||
@@ -0,0 +1,54 @@
|
||||
.. _serve-tensorflow-tutorial:
|
||||
|
||||
Keras and Tensorflow Tutorial
|
||||
=============================
|
||||
|
||||
In this guide, we will train and deploy a simple Tensorflow neural net.
|
||||
In particular, we show:
|
||||
|
||||
- How to load the model from file system in your RayServe definition
|
||||
- How to parse the JSON request and evaluated in Tensorflow
|
||||
|
||||
Please see the :ref:`overview <rayserve-overview>` to learn more general information about RayServe.
|
||||
|
||||
RayServe makes it easy to deploy models from :ref:`all popular frameworks <serve_frameworks>`.
|
||||
However, for this tutorial, we use Tensorflow 2 and Keras. Please make sure you have
|
||||
Tensorflow 2 installed.
|
||||
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install "tensorflow>=2.0"
|
||||
|
||||
Let's import RayServe and some other helpers.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_tensorflow.py
|
||||
:start-after: __doc_import_begin__
|
||||
:end-before: __doc_import_end__
|
||||
|
||||
We will train a simple MNIST model using Keras.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_tensorflow.py
|
||||
:start-after: __doc_train_model_begin__
|
||||
:end-before: __doc_train_model_end__
|
||||
|
||||
Services are just defined as normal classes with ``__init__`` and ``__call__`` methods.
|
||||
The ``__call__`` method will be invoked per request.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_tensorflow.py
|
||||
:start-after: __doc_define_servable_begin__
|
||||
:end-before: __doc_define_servable_end__
|
||||
|
||||
Now that we've defined our services, let's deploy the model to RayServe. We will
|
||||
define an :ref:`endpoint <serve-endpoint>` for the route representing the digit classifier task, a
|
||||
:ref:`backend <serve-backend>` correspond the physical implementation, and connect them together.
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_tensorflow.py
|
||||
:start-after: __doc_deploy_begin__
|
||||
:end-before: __doc_deploy_end__
|
||||
|
||||
Let's query it!
|
||||
|
||||
.. literalinclude:: ../../../../python/ray/serve/examples/doc/tutorial_tensorflow.py
|
||||
:start-after: __doc_query_begin__
|
||||
:end-before: __doc_query_end__
|
||||
Reference in New Issue
Block a user