From 1a1ddc74c4d08a8edd43ebf75b5229e8629b63c5 Mon Sep 17 00:00:00 2001 From: Simon Mo Date: Mon, 15 Jun 2020 18:47:59 -0700 Subject: [PATCH] [Serve] Add package reference and links keyword to docstring (#8955) --- doc/source/index.rst | 1 + doc/source/serve/advanced.rst | 12 +++++----- doc/source/serve/deployment.rst | 11 +++++---- doc/source/serve/key-concepts.rst | 10 ++++---- doc/source/serve/package-ref.rst | 40 +++++++++++++++++++++++++++++++ 5 files changed, 59 insertions(+), 15 deletions(-) create mode 100644 doc/source/serve/package-ref.rst diff --git a/doc/source/index.rst b/doc/source/index.rst index 425edafe7..354410634 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -193,6 +193,7 @@ Getting Involved serve/tutorials/index.rst serve/deployment.rst serve/advanced.rst + serve/package-ref.rst .. toctree:: :maxdepth: -1 diff --git a/doc/source/serve/advanced.rst b/doc/source/serve/advanced.rst index 25cd0e8f7..9980ff757 100644 --- a/doc/source/serve/advanced.rst +++ b/doc/source/serve/advanced.rst @@ -102,8 +102,8 @@ Splitting Traffic Between Backends ================================== At times it may be useful to expose a single endpoint that is served by multiple backends. -You can do this by splitting the traffic for an endpoint between backends using ``set_traffic``. -When calling ``set_traffic``, you provide a dictionary of backend name to a float value that will be used to randomly route that portion of traffic (out of a total of 1.0) to the given backend. +You can do this by splitting the traffic for an endpoint between backends using :mod:`set_traffic `. +When calling :mod:`set_traffic `, you provide a dictionary of backend name to a float value that will be used to randomly route that portion of traffic (out of a total of 1.0) to the given backend. For example, here we split traffic 50/50 between two backends: .. code-block:: python @@ -120,7 +120,7 @@ Please see :ref:`session-affinity` for details on how to ensure that clients or A/B Testing ----------- -``set_traffic`` can be used to implement A/B testing by having one backend serve the majority of traffic while a fraction is routed to a second model: +:mod:`set_traffic ` can be used to implement A/B testing by having one backend serve the majority of traffic while a fraction is routed to a second model: .. code-block:: python @@ -143,7 +143,7 @@ A/B Testing Incremental Rollout ------------------- -``set_traffic`` can also be used to implement incremental rollout. +:mod:`set_traffic ` can also be used to implement incremental rollout. Here, we want to replace an existing backend with a new implementation by gradually increasing the proportion of traffic that it serves. In the example below, we do this repeatedly in one script, but in practice this would likely happen over time across multiple scripts. @@ -221,7 +221,7 @@ See :doc:`deployment` for information about how to deploy serve. How do I delete backends and endpoints? --------------------------------------- -To delete a backend, you can use `serve.delete_backend`. +To delete a backend, you can use :mod:`serve.delete_backend `. Note that the backend must not be use by any endpoints in order to be delete. Once a backend is deleted, its tag can be reused. @@ -230,7 +230,7 @@ Once a backend is deleted, its tag can be reused. serve.delete_backend("simple_backend") -To delete a endpoint, you can use `serve.delete_endpoint`. +To delete a endpoint, you can use :mod:`serve.delete_endpoint `. Note that the endpoint will no longer work and return a 404 when queried. Once a endpoint is deleted, its tag can be reused. diff --git a/doc/source/serve/deployment.rst b/doc/source/serve/deployment.rst index 0a263a7b8..40e9debb5 100644 --- a/doc/source/serve/deployment.rst +++ b/doc/source/serve/deployment.rst @@ -56,14 +56,16 @@ Creating a Model and Serving it ------------------------------- In the following snippet we will complete two things: + 1. Define a servable model by instantiating a class and defining the ``__call__`` method. -2. Connect to our running Ray cluster(``ray.init(...)``) and then start or connect to the Ray Serve instance on that cluster(``serve.init(...)``). +2. Connect to our running Ray cluster(``ray.init(...)``) and then start or connect to the Ray Serve instance on that + cluster(:mod:`serve.init(...) `). You can see that defining the model is straightforward and simple, we're simply instantiating the model like we might a typical Python class. -Configuring our model to accept traffic is specified via ``.set_traffic`` after we created +Configuring our model to accept traffic is specified via :mod:`.set_traffic ` after we created a backend in serve for our model (and versioned it with a string). .. literalinclude:: ../../../python/ray/serve/examples/doc/tutorial_deploy.py @@ -72,7 +74,8 @@ a backend in serve for our model (and versioned it with a string). What serve does when we run this code is store the model as a Ray actor and route traffic to it as the endpoint is queried, in this case over HTTP. -Note that in order for this endpoint to be accessible from other machines, we need to specify ``http_host="0.0.0.0"`` in ``serve.init`` like we did here. +Note that in order for this endpoint to be accessible from other machines, we +need to specify ``http_host="0.0.0.0"`` in :mod:`serve.init ` like we did here. Now let's query our endpoint to see the result. @@ -278,7 +281,7 @@ To learn more, in general, about Ray Clusters see :doc:`../cluster-index`. Deploying Multiple Serve Instaces on a Single Ray Cluster --------------------------------------------------------- -You can run multiple serve instances on the same Ray cluster by providing a ``name`` in ``serve.init()``. +You can run multiple serve instances on the same Ray cluster by providing a ``name`` in :mod:`serve.init() `. .. code-block:: python diff --git a/doc/source/serve/key-concepts.rst b/doc/source/serve/key-concepts.rst index 11d99d53e..2ffed4517 100644 --- a/doc/source/serve/key-concepts.rst +++ b/doc/source/serve/key-concepts.rst @@ -19,9 +19,9 @@ Backends Backends define the implementation of your business logic or models that will handle requests when queries come in to :ref:`serve-endpoint`. To define a backend, first you must define the "handler" or the business logic you'd like to respond with. The handler should take as input a `Flask Request object `_ and return any JSON-serializable object as output. -A backend is defined using ``serve.create_backend``, and the implementation can be defined as either a function or a class. +A backend is defined using :mod:`serve.create_backend `, and the implementation can be defined as either a function or a class. Use a function when your response is stateless and a class when you might need to maintain some state (like a model). -When using a class, you can specify arguments to be passed to the constructor in ``serve.create_backend``, shown below. +When using a class, you can specify arguments to be passed to the constructor in :mod:`serve.create_backend `, shown below. A backend consists of a number of *replicas*, which are individual copies of the function or class that are started in separate worker processes. @@ -67,7 +67,7 @@ Endpoints While backends define the implementation of your request handling logic, endpoints allow you to expose them via HTTP. Endpoints are "logical" and can have one or multiple backends that serve requests to them To create an endpoint, we simply need to specify a name for the endpoint, the name of a backend to handle requests to the endpoint, and the route and methods where it will be accesible. -By default endpoints are serviced only by the backend provided to ``serve.create_endpoint``, but in some cases you may want to specify multiple backends for an endpoint, e.g., for A/B testing or incremental rollout. +By default endpoints are serviced only by the backend provided to :mod:`serve.create_endpoint `, but in some cases you may want to specify multiple backends for an endpoint, e.g., for A/B testing or incremental rollout. For information on how to do this, please see :ref:`serve-split-traffic`. .. code-block:: python @@ -82,14 +82,14 @@ We can query the model to verify that it's working. import requests print(requests.get("http://127.0.0.1:8000/simple").text) -To view all of the existing endpoints that have created, use `serve.list_endpoints`. +To view all of the existing endpoints that have created, use :mod:`serve.list_endpoints `. .. code-block:: python >>> serve.list_endpoints() {'simple_endpoint': {'route': '/simple', 'methods': ['GET'], 'traffic': {}}} -You can also delete an endpoint using ``serve.delete_endpoint``. +You can also delete an endpoint using :mod:`serve.delete_endpoint `. Endpoints and backends are independent, so deleting an endpoint will not delete its backends. However, an endpoint must be deleted in order to delete the backends that serve its traffic. diff --git a/doc/source/serve/package-ref.rst b/doc/source/serve/package-ref.rst new file mode 100644 index 000000000..5fb94aeeb --- /dev/null +++ b/doc/source/serve/package-ref.rst @@ -0,0 +1,40 @@ +Package Reference +================= + +Basic APIs +---------- +.. autofunction:: ray.serve.init +.. autofunction:: ray.serve.create_backend +.. autofunction:: ray.serve.create_endpoint + + +APIs for Managing Endpoints +--------------------------- +.. autofunction:: ray.serve.list_endpoints +.. autofunction:: ray.serve.delete_endpoint +.. autofunction:: ray.serve.set_traffic + + +APIs for Managing Backends +-------------------------- +.. autofunction:: ray.serve.list_backends +.. autofunction:: ray.serve.delete_backend +.. autofunction:: ray.serve.get_backend_config +.. autofunction:: ray.serve.update_backend_config + +Advanced APIs +------------- + +``serve.get_handle`` enables calling endpoints from Python. + +.. autofunction:: ray.serve.get_handle +.. autoclass:: ray.serve.handle.RayServeHandle + +``serve.stat`` queries Ray Serve's built-in metric monitor. +.. autofunction:: ray.serve.stat + + +``serve.accept_batch`` marks your backend API does accept list of input instead +of just single input. +.. autofunction:: ray.serve.accept_batch +