wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-12 12:20:11 +08:00
Code Issues Packages Projects Releases Wiki Activity

[serve] Optionally namespace serve clusters (#8447)

Browse Source
This commit is contained in:
Edward Oakes
2020-05-17 00:14:42 -05:00
committed by GitHub
parent 67c01455fe
commit fb23bd6fc0
10 changed files with 163 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/rayserve/overview.rst
+24
View File
@@ -246,6 +246,30 @@ The shard key can either be specified via the X-SERVE-SHARD-KEY HTTP header or `
handle = serve.get_handle("api_endpoint")
handler.options(shard_key=session_id).remote(args)
Running Multiple Serve Clusters on one Ray Cluster
++++++++++++++++++++++++++++++++++++++++++++++++++
You can run multiple serve clusters on the same Ray cluster by providing a ``cluster_name`` to ``serve.init()``.
.. code-block:: python
# Create a first cluster whose HTTP server listens on 8000.
serve.init(cluster_name="cluster1", http_port=8000)
serve.create_endpoint("counter1", "/increment")
# Create a second cluster whose HTTP server listens on 8001.
serve.init(cluster_name="cluster2", http_port=8001)
serve.create_endpoint("counter1", "/increment")
# Create a backend that will be served on the second cluster.
serve.create_backend("counter1", function)
serve.set_traffic("counter1", {"counter1": 1.0})
# Switch back the the first cluster and create the same backend on it.
serve.init(cluster_name="cluster1")
serve.create_backend("counter1", function)
serve.set_traffic("counter1", {"counter1": 1.0})
Other Resources
---------------