mirror of
https://github.com/wassname/ray.git
synced 2026-07-27 11:26:41 +08:00
[Serve] Support basic Starlette response types (#12811)
This commit is contained in:
@@ -37,7 +37,7 @@ Since Serve is built on Ray, it also allows you to scale to many machines, in yo
|
||||
Installation
|
||||
============
|
||||
|
||||
Ray Serve supports Python versions 3.6 and higher. To install Ray Serve:
|
||||
Ray Serve supports Python versions 3.6 through 3.8. To install Ray Serve:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
|
||||
@@ -19,7 +19,11 @@ Backends
|
||||
Backends define the implementation of your business logic or models that will handle requests when queries come in to :ref:`serve-endpoint`.
|
||||
In order to support seamless scalability backends can have many replicas, which are individual processes running in the Ray cluster to handle requests.
|
||||
To define a backend, first you must define the "handler" or the business logic you'd like to respond with.
|
||||
The handler should take as input a `Flask Request object <https://flask.palletsprojects.com/en/1.1.x/api/?highlight=request#flask.Request>`_ and return any JSON-serializable object as output.
|
||||
The handler should take as input a `Flask Request object <https://flask.palletsprojects.com/en/1.1.x/api/?highlight=request#flask.Request>`_.
|
||||
The handler should return any JSON-serializable object as output. For a more customizable response type, the handler may return a
|
||||
`Starlette Response object <https://www.starlette.io/responses/>`_.
|
||||
In the future, Ray Serve will support `Starlette Request objects <https://www.starlette.io/requests/>`_ as input as well.
|
||||
|
||||
A backend is defined using :mod:`client.create_backend <ray.serve.api.Client.create_backend>`, and the implementation can be defined as either a function or a class.
|
||||
Use a function when your response is stateless and a class when you might need to maintain some state (like a model).
|
||||
When using a class, you can specify arguments to be passed to the constructor in :mod:`client.create_backend <ray.serve.api.Client.create_backend>`, shown below.
|
||||
|
||||
Reference in New Issue
Block a user