Add docstring for find_redis_address (#11884)

This commit is contained in:
dHannasch
2020-11-11 11:24:36 -07:00
committed by GitHub
parent 291c172d83
commit 396ae0b7c2
+48
View File
@@ -122,6 +122,54 @@ def new_port():
def find_redis_address(address=None):
"""
Currently, this extracts the deprecated --redis-address from the command
that launched the raylet running on this node, if any. Anyone looking to
edit this function should be warned that these commands look like, for
example:
/usr/local/lib/python3.8/dist-packages/ray/core/src/ray/raylet/raylet
--redis_address=123.456.78.910 --node_ip_address=123.456.78.910
--raylet_socket_name=... --store_socket_name=... --object_manager_port=0
--min_worker_port=10000 --max_worker_port=10999 --node_manager_port=58578
--redis_port=6379 --num_initial_workers=8 --maximum_startup_concurrency=8
--static_resource_list=node:123.456.78.910,1.0,object_store_memory,66
--config_list=plasma_store_as_thread,True
--python_worker_command=/usr/bin/python
/usr/local/lib/python3.8/dist-packages/ray/workers/default_worker.py
--redis-address=123.456.78.910:6379
--node-ip-address=123.456.78.910 --node-manager-port=58578
--object-store-name=... --raylet-name=...
--config-list=plasma_store_as_thread,True --temp-dir=/tmp/ray
--metrics-agent-port=41856 --redis-password=[MASKED]
--java_worker_command= --cpp_worker_command= --redis_password=[MASKED]
--temp_dir=/tmp/ray --session_dir=... --metrics-agent-port=41856
--metrics_export_port=64229
--agent_command=/usr/bin/python
-u /usr/local/lib/python3.8/dist-packages/ray/new_dashboard/agent.py
--redis-address=123.456.78.910:6379 --metrics-export-port=64229
--dashboard-agent-port=41856 --node-manager-port=58578
--object-store-name=... --raylet-name=... --temp-dir=/tmp/ray
--log-dir=/tmp/ray/session_2020-11-08_14-29-07_199128_278000/logs
--redis-password=[MASKED] --object_store_memory=5037192806
--plasma_directory=/tmp
Longer arguments are elided with ... but all arguments from this instance
are included, to provide a sense of what is in these.
Indeed, we had to pull --redis-address to the front of each call to make
this readable.
As you can see, this is very long and complex, which is why we can't simply
extract all the the arguments using regular expressions and present a dict
as if we never lost track of these arguments, for example.
Picking out --redis-address below looks like it might grab the wrong thing,
but double-checking that we're finding the correct process by checking that
the contents look like we expect would probably be prone to choking in
unexpected ways.
Notice that --redis-address appears twice. This is not a copy-paste error;
this is the reason why the for loop below attempts to pick out every
appearance of --redis-address.
The --redis-address here is what is now called the --address, but it
appears in the default_worker.py and agent.py calls as --redis-address.
"""
pids = psutil.pids()
redis_addresses = set()
for pid in pids: