[Serve] HTTPOptions for deployment modes (#13142)

doc/source/serve/advanced.rst

@@ -405,3 +405,21 @@ backend based on a class that is installed in the Python environment that
 the workers will run in. Example:
 
 .. literalinclude:: ../../../python/ray/serve/examples/doc/imported_backend.py
+
+Configuring HTTP Server Locations
+=================================
+
+By default, Ray Serve starts only one HTTP on the head node of the Ray cluster.
+You can configure this behavior using the ``http_options={"location": ...}`` flag
+in :mod:`serve.start <ray.serve.start>`:
+
+- "HeadOnly": start one HTTP server on the head node. Serve
+  assumes the head node is the node you executed serve.start
+  on. This is the default.
+- "EveryNode": start one HTTP server per node.
+- "NoServer" or ``None``: disable HTTP server.
+
+.. note::
+   Using the "EveryNode" option, you can point a cloud load balancer to the
+   instance group of Ray cluster to achieve high availability of Serve's HTTP
+   proxies.

doc/source/serve/deployment.rst

@@ -69,7 +69,7 @@ a backend in serve for our model (and versioned it with a string).
 What serve does when we run this code is store the model as a Ray actor
 and route traffic to it as the endpoint is queried, in this case over HTTP.
 Note that in order for this endpoint to be accessible from other machines, we
-need to specify ``http_host="0.0.0.0"`` in :mod:`serve.start <ray.serve.start>` like we did here.
+need to specify ``http_options={"host": "0.0.0.0"}`` in :mod:`serve.start <ray.serve.start>` like we did here.
 
 Now let's query our endpoint to see the result.
 
@@ -225,7 +225,7 @@ With the cluster now running, we can run a simple script to start Ray Serve and
     # Connect to the running Ray cluster.
     ray.init(address="auto")
     # Bind on 0.0.0.0 to expose the HTTP server on external IPs.
-    client = serve.start(http_host="0.0.0.0")
+    client = serve.start(http_options={"host": "0.0.0.0"})
 
     def hello():
         return "hello world"

doc/source/serve/faq.rst

@@ -96,10 +96,10 @@ You can follow the same pattern for other Starlette middlewares.
     from starlette.middleware.cors import CORSMiddleware
 
     client = serve.start(
-        http_middlewares=[
+        http_options={"middlewares": [
             Middleware(
                 CORSMiddleware, allow_origins=["*"], allow_methods=["*"])
-        ])
+        ]})
 
 
 .. _serve-handle-explainer: