[Serve] [Doc] Add existing web server integration ServeHandle tutorial (#13127)

doc/source/serve/tutorials/index.rst

@@ -2,7 +2,7 @@
 Tutorials
 =========
 
-Below are a list of tutorials that you can use to learn more about the different pieces of
+Below is a list of tutorials that you can use to learn more about the different pieces of
 Ray Serve functionality and how to integrate different modeling frameworks.
 
 .. toctree::
@@ -14,7 +14,9 @@ Ray Serve functionality and how to integrate different modeling frameworks.
    pytorch.rst
    sklearn.rst
    batch.rst
+   web-server-integration.rst
 
 
 Other Topics:
+
 - :doc:`../deployment`

doc/source/serve/tutorials/web-server-integration.rst

@@ -0,0 +1,81 @@
+.. _serve-web-server-integration-tutorial:
+
+Integration with Existing Web Servers
+=====================================
+
+In this guide, you will learn how to use Ray Serve to scale up your existing web application.  The key feature of Ray Serve that makes this possible is the Python-native :ref:`servehandle-api`, which allows you keep using your same Python web server while offloading your heavy computation to Ray Serve.
+
+We give two examples, one using a `FastAPI <https://fastapi.tiangolo.com/>`__ web server and another using an `AIOHTTP <https://docs.aiohttp.org/en/stable/>`__ web server, but the same approach will work with any Python web server.
+
+
+Scaling Up a FastAPI Application
+--------------------------------
+
+For this example, you must have either `Pytorch <https://pytorch.org/>`_ or `Tensorflow <https://www.tensorflow.org/>`_ installed, as well as `Huggingface Transformers <https://github.com/huggingface/transformers>`_ and `FastAPI <https://fastapi.tiangolo.com/>`_.  For example:
+
+.. code-block:: bash
+
+  pip install "ray[serve]" tensorflow transformers fastapi
+
+Here’s a simple FastAPI web server. It uses Huggingface Transformers to auto-generate text based on a short initial input using `OpenAI’s GPT-2 model <https://openai.com/blog/better-language-models/>`_.
+
+.. literalinclude:: ../../../../python/ray/serve/examples/doc/fastapi/fastapi.py
+
+To scale this up, we define a Ray Serve backend containing our text model and call it from Python using a ServeHandle:
+
+.. literalinclude:: ../../../../python/ray/serve/examples/doc/fastapi/servehandle_fastapi.py
+
+To run this example, save it as ``main.py`` and then in the same directory, run the following commands to start a local Ray cluster on your machine and run the FastAPI application:
+
+.. code-block:: bash
+
+  ray start --head
+  uvicorn main:app
+
+Now you can query your web server, for example by running the following in another terminal:
+
+.. code-block:: bash
+
+  curl "http://127.0.0.1:8000/generate?query=Hello%20friend%2C%20how"
+  
+The terminal should then print the generated text:
+
+.. code-block:: bash
+
+  [{"generated_text":"Hello friend, how's your morning?\n\nSven: Thank you.\n\nMRS. MELISSA: I feel like it really has done to you.\n\nMRS. MELISSA: The only thing I"}]%
+
+To clean up the Ray cluster, run ``ray stop`` in the terminal.
+
+.. tip::
+  According to the backend configuration parameter ``num_replicas``, Ray Serve will place multiple replicas of your model across multiple CPU cores and multiple machines (provided you have :ref:`started a multi-node Ray cluster <cluster-index>`), which will correspondingly multiply your throughput.
+
+Scaling Up an AIOHTTP Application
+---------------------------------
+
+In this section, we'll integrate Ray Serve with an `AIOHTTP <https://docs.aiohttp.org/en/stable/>`_ web server run using `Gunicorn <https://gunicorn.org/>`_.  You'll need to install AIOHTTP and gunicorn with the command ``pip install aiohttp gunicorn``.
+
+First, here is the script that deploys Ray Serve:
+
+.. literalinclude:: ../../../../python/ray/serve/examples/doc/aiohttp/aiohttp_deploy_serve.py
+
+Next is the script that defines the AIOHTTP server:
+
+.. literalinclude:: ../../../../python/ray/serve/examples/doc/aiohttp/aiohttp_app.py
+
+Here's how to run this example:
+
+1. Run ``ray start --head`` to start a local Ray cluster in the background.
+
+2. In the directory where the example files are saved, run ``python deploy_serve.py`` to deploy our Ray Serve endpoint.  
+
+.. note::
+  Because we have omitted the keyword argument ``route`` in ``client.create_endpoint()``, our endpoint will not be exposed over HTTP by Ray Serve.
+
+3. Run ``gunicorn aiohttp_app:app --worker-class aiohttp.GunicornWebWorker --bind localhost:8001`` to start the AIOHTTP app using gunicorn. We bind to port 8001 because the Ray Dashboard is already using port 8000 by default.
+
+.. tip::
+  You can change the Ray Dashboard port with the command ``ray start --dashboard-port XXXX``.
+
+4. To test out the server, run ``curl localhost:8001/dummy-model``.  This should output ``Model received data: dummy input``.
+
+5. For cleanup, you can press Ctrl-C to stop the Gunicorn server, and run ``ray stop`` to stop the background Ray cluster.