mirror of
https://github.com/wassname/ray.git
synced 2026-06-27 16:31:25 +08:00
Robert Nishihara
658c14282c
* Remove legacy Ray code. * Fix cmake and simplify monitor. * Fix linting * Updates * Fix * Implement some methods. * Remove more plasma manager references. * Fix * Linting * Fix * Fix * Make sure class IDs are strings. * Some path fixes * Fix * Path fixes and update arrow * Fixes. * linting * Fixes * Java fixes * Some java fixes * TaskLanguage -> Language * Minor * Fix python test and remove unused method signature. * Fix java tests * Fix jenkins tests * Remove commented out code.
86 lines
3.3 KiB
ReStructuredText
86 lines
3.3 KiB
ReStructuredText
Temporary Files
|
||
===============
|
||
|
||
Ray will produce some temporary files during running.
|
||
They are useful for logging, debugging & sharing object store with other programs.
|
||
|
||
Location of Temporary Files
|
||
---------------------------
|
||
|
||
First we introduce the concept of a session of Ray.
|
||
|
||
A session contains a set of processes. A session is created by executing
|
||
``ray start`` command or call ``ray.init()`` in a Python script and ended by
|
||
executing ``ray stop`` or call ``ray.shutdown()``.
|
||
|
||
For each session, Ray will create a *root temporary directory* to place all its
|
||
temporary files. The path is ``/tmp/ray/session_{datetime}_{pid}`` by default.
|
||
The pid belongs to the startup process (the process calling ``ray.init()`` or
|
||
the Ray process executed by a shell in ``ray start``).
|
||
You can sort by their names to find the latest session.
|
||
|
||
You are allowed to change the *root temporary directory* in one of these ways:
|
||
|
||
* Pass ``--temp-dir={your temp path}`` to ``ray start``
|
||
* Specify ``temp_dir`` when call ``ray.init()``
|
||
|
||
You can also use ``default_worker.py --temp-dir={your temp path}`` to
|
||
start a new worker with given *root temporary directory*.
|
||
|
||
The *root temporary directory* you specified will be given as it is,
|
||
without pids or datetime attached.
|
||
|
||
Layout of Temporary Files
|
||
-------------------------
|
||
|
||
A typical layout of temporary files could look like this:
|
||
|
||
.. code-block:: text
|
||
|
||
/tmp
|
||
└── ray
|
||
└── session_{datetime}_{pid}
|
||
├── logs
|
||
│ ├── log_monitor.err
|
||
│ ├── log_monitor.out
|
||
│ ├── monitor.err
|
||
│ ├── monitor.out
|
||
│ ├── plasma_store_0.err # array of plasma stores' outputs
|
||
│ ├── plasma_store_0.out
|
||
│ ├── raylet_0.err # array of raylets' outputs. Control it with `--no-redirect-worker-output` (in Ray's command line) or `redirect_worker_output` (in ray.init())
|
||
│ ├── raylet_0.out
|
||
│ ├── redis-shard_0.err # array of redis shards' outputs
|
||
│ ├── redis-shard_0.out
|
||
│ ├── redis.err # redis
|
||
│ ├── redis.out
|
||
│ ├── webui.err # ipython notebook web ui
|
||
│ ├── webui.out
|
||
│ ├── worker-{worker_id}.err # redirected output of workers
|
||
│ ├── worker-{worker_id}.out
|
||
│ └── {other workers}
|
||
├── ray_ui.ipynb # ipython notebook file
|
||
└── sockets # for logging
|
||
├── plasma_store
|
||
└── raylet # this could be deleted by Ray's shutdown cleanup.
|
||
|
||
|
||
Plasma Object Store Socket
|
||
--------------------------
|
||
|
||
Plasma object store sockets can be used to share objects with other programs using Apache Arrow.
|
||
|
||
You are allowed to specify the plasma object store socket in one of these ways:
|
||
|
||
* Pass ``--plasma-store-socket-name={your socket path}`` to ``ray start``
|
||
* Specify ``plasma_store_socket_name`` when call ``ray.init()``
|
||
|
||
The path you specified will be given as it is without being affected any other paths.
|
||
|
||
Notes
|
||
-----
|
||
|
||
Temporary file policies are defined in ``python/ray/tempfile_services.py``.
|
||
|
||
Currently, we keep ``/tmp/ray`` as the default directory for temporary data files of RLlib as before.
|
||
It is not very reasonable and could be changed later.
|