wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-07-27 11:26:41 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Logging] Remove per worker job log file / support worker log rotation (#11927)

Browse Source 
* In progress.

* MVP done.

* In Progress.

* Remove unnecessay code.

* Fix some issues.

* Fix test failures.

* Addressed code review + fix object spilling test failure.
This commit is contained in:
SangBin Cho
2020-11-16 11:29:43 -08:00
committed by GitHub
parent b6b54f1c81
commit f56d7c1a76
11 changed files with 215 additions and 254 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/ray/_raylet.pyx
-14
View File
@@ -326,19 +326,6 @@ cdef prepare_args(
CCoreWorkerProcess.GetCoreWorker().GetRpcAddress())))
def switch_worker_log_if_needed(worker, next_job_id):
if worker.mode != ray.WORKER_MODE:
return
if (worker.current_logging_job_id is None) or \
(worker.current_logging_job_id != next_job_id):
job_stdout_path, job_stderr_path = (
worker.node.get_job_redirected_log_file(
worker.worker_id, next_job_id.binary())
)
ray.worker.set_log_file(job_stdout_path, job_stderr_path)
worker.current_logging_job_id = next_job_id
cdef execute_task(
CTaskType task_type,
const c_string name,
@@ -472,7 +459,6 @@ cdef execute_task(
with core_worker.profile_event(b"task:execute"):
task_exception = True
try:
switch_worker_log_if_needed(worker, job_id)
with ray.worker._changeproctitle(title, next_title):
outputs = function_executor(*args, **kwargs)
task_exception = False