mirror of
https://github.com/wassname/ray.git
synced 2026-08-11 11:24:51 +08:00
Add "beta" documentation for enabling object spilling manually (#13047)
This commit is contained in:
@@ -392,3 +392,39 @@ To get information about the current available resource capacity of your cluster
|
||||
|
||||
.. autofunction:: ray.available_resources
|
||||
:noindex:
|
||||
|
||||
Object Spilling
|
||||
---------------
|
||||
|
||||
Ray 1.2.0+ has *beta* support for spilling objects to external storage once the capacity
|
||||
of the object store is used up. Please file a `GitHub issue <https://github.com/ray-project/ray/issues/>`__
|
||||
if you encounter any problems with this new feature. Eventually, object spilling will be
|
||||
enabled by default, but for now you need to enable it manually:
|
||||
|
||||
To enable object spilling to the local filesystem (single node clusters only):
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ray.init(
|
||||
_system_config={
|
||||
"automatic_object_spilling_enabled": True,
|
||||
"object_spilling_config": json.dumps(
|
||||
{"type": "filesystem", "params": {"directory_path": "/tmp/spill"}},
|
||||
)
|
||||
},
|
||||
)
|
||||
|
||||
To enable object spilling to remote storage (any URI supported by `smart_open <https://pypi.org/project/smart-open/>`__):
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ray.init(
|
||||
_system_config={
|
||||
"automatic_object_spilling_enabled": True,
|
||||
"max_io_workers": 4, # More IO workers for remote storage.
|
||||
"min_spilling_size": 100 * 1024 * 1024, # Spill at least 100MB at a time.
|
||||
"object_spilling_config": json.dumps(
|
||||
{"type": "smart_open", "params": {"uri": "s3:///bucket/path"}},
|
||||
)
|
||||
},
|
||||
)
|
||||
|
||||
@@ -417,6 +417,8 @@ actors.
|
||||
to the object ref returned by the put exists. This only applies to the specific
|
||||
ref returned by put, not refs in general or copies of that refs.
|
||||
|
||||
See also: `object spilling <advanced.html#object-spilling>`__.
|
||||
|
||||
Remote Classes (Actors)
|
||||
-----------------------
|
||||
|
||||
|
||||
Reference in New Issue
Block a user