[xray] Implement timeline and profiling API. (#2306)

* Add profile table and store profiling information there.

* Code for dumping timeline.

* Improve color scheme.

* Push timeline events on driver only for raylet.

* Improvements to profiling and timeline visualization

* Some linting

* Small fix.

* Linting

* Propagate node IP address through profiling events.

* Fix test.

* object_id.hex() should return byte string in python 2.

* Include gcs.fbs in node_manager.fbs.

* Remove flatbuffer definition duplication.

* Decode to unicode in Python 3 and bytes in Python 2.

* Minor

* Submit profile events in a batch. Revert some CMake changes.

* Fix

* Workaround test failure.

* Fix linting

* Linting

* Don't return anything from chrome_tracing_dump when filename is provided.

* Remove some redundancy from profile table.

* Linting

* Move TODOs out of docstring.

* Minor
This commit is contained in:
Robert Nishihara
2018-07-04 23:23:48 -07:00
committed by Philipp Moritz
parent 8e687cbc98
commit b90e551b41
27 changed files with 777 additions and 147 deletions
+14 -2
View File
@@ -11,6 +11,18 @@ import subprocess
import sys
# This is duplicated from ray.utils so that we do not have to introduce a
# dependency on Ray to run this file.
def decode(byte_str):
"""Make this unicode in Python 3, otherwise leave it as bytes."""
if not isinstance(byte_str, bytes):
raise ValueError("The argument must be a bytes object.")
if sys.version_info >= (3, 0):
return byte_str.decode("ascii")
else:
return byte_str
def wait_for_output(proc):
"""This is a convenience method to parse a process's stdout and stderr.
@@ -27,7 +39,7 @@ def wait_for_output(proc):
# NOTE(rkn): This try/except block is here because I once saw an
# exception raised here and want to print more information if that
# happens again.
stdout_data = stdout_data.decode("ascii")
stdout_data = decode(stdout_data)
except UnicodeDecodeError:
raise Exception("Failed to decode stdout_data:", stdout_data)
@@ -36,7 +48,7 @@ def wait_for_output(proc):
# NOTE(rkn): This try/except block is here because I once saw an
# exception raised here and want to print more information if that
# happens again.
stderr_data = stderr_data.decode("ascii")
stderr_data = decode(stderr_data)
except UnicodeDecodeError:
raise Exception("Failed to decode stderr_data:", stderr_data)