wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-09-09 11:32:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

[docs] Note that the printed IP address can be incorrect. (#11804)

Browse Source 
* If the head node is on a subnet with NAT, then you will need a different IP address.

* Specify what you are checking firewall settings and network configuration *for*.

* reword following @amogkam

* Give the full error message.
This commit is contained in:
dHannasch
2020-11-04 13:48:03 -08:00
committed by GitHub
parent ebdf8ba3fa
commit 6147b6a1a3
1 changed files with 53 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/cluster/index.rst
+53 -4
View File
@@ -102,6 +102,13 @@ The command will print out the address of the Redis server that was started
``<address>`` with the value printed by the command on the head node (it
should look something like ``123.45.67.89:6379``).
Note that if your compute nodes are on their own subnetwork with Network
Address Translation, to connect from a regular machine outside that subnetwork,
the command printed by the head node will not work. You need to find the
address that will reach the head node from the second machine. If the head node
has a domain address like compute04.berkeley.edu, you can simply use that in
place of an IP address and rely on the DNS.
.. code-block:: bash
$ ray start --address=<address> --redis-password='<password>'
@@ -115,11 +122,53 @@ should look something like ``123.45.67.89:6379``).
If you wish to specify that a machine has 10 CPUs and 1 GPU, you can do this
with the flags ``--num-cpus=10`` and ``--num-gpus=1``. See the :ref:`Configuration <configuring-ray>` page for more information.
If you see ``Unable to connect to Redis. If the Redis instance is on a
different machine, check that your firewall is configured properly.``,
this means the ``--port`` is inaccessible at the given IP address (because, for
example, the head node is not actually running Ray, or you have the wrong IP
address).
If you see ``Ray runtime started.``, then the node successfully connected to
the ``<address>``. If the ``<address>`` is inaccessible (because, for example,
the head node is not actually running), then you will get an error such as
``Unable to connect to Redis. If the Redis instance is on a different machine,
check that your firewall is configured properly.``
the IP address at the ``--port``. You should now be able to connect to the
cluster with ``ray.init(address='auto')``.
If ``ray.init(address='auto')`` keeps repeating
``redis_context.cc:303: Failed to connect to Redis, retrying.``, then the node
is failing to connect to some other port(s) besides the main port.
.. code-block:: bash
If connection fails, check your firewall settings and network configuration.
If the connection fails, to check whether each port can be reached from a node,
you can use a tool such as ``nmap`` or ``nc``.
.. code-block:: bash
$ nmap -sV --reason -p $PORT $HEAD_ADDRESS
Nmap scan report for compute04.berkeley.edu (123.456.78.910)
Host is up, received echo-reply ttl 60 (0.00087s latency).
rDNS record for 123.456.78.910: compute04.berkeley.edu
PORT STATE SERVICE REASON VERSION
6379/tcp open redis syn-ack ttl 60 Redis key-value store
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
$ nc -vv -z $HEAD_ADDRESS $PORT
Connection to compute04.berkeley.edu 6379 port [tcp/*] succeeded!
If the node cannot access that port at that IP address, you might see
.. code-block:: bash
$ nmap -sV --reason -p $PORT $HEAD_ADDRESS
Nmap scan report for compute04.berkeley.edu (123.456.78.910)
Host is up (0.0011s latency).
rDNS record for 123.456.78.910: compute04.berkeley.edu
PORT STATE SERVICE REASON VERSION
6379/tcp closed redis reset ttl 60
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
$ nc -vv -z $HEAD_ADDRESS $PORT
nc: connect to compute04.berkeley.edu port 6379 (tcp) failed: Connection refused
Stopping Ray
~~~~~~~~~~~~