Limit number of concurrent workers started by hardware concurrency. (#2753)

* Limit number of concurrent workers started by hardware concurrency.

* Check if std::thread::hardware_concurrency() returns 0.

* Pass in max concurrency from Python.

* Fix Java call to startRaylet.

* Fix typo

* Remove unnecessary cast.

* Fix linting.

* Cleanups on Java side.

* Comment back in actor test.

* Require maximum_startup_concurrency to be at least 1.

* Fix linting and test.

* Improve documentation.

* Fix typo.

python/ray/services.py

@@ -1008,6 +1008,11 @@ def start_raylet(redis_address,
 
     static_resources = check_and_update_resources(resources, True)
 
+    # Limit the number of workers that can be started in parallel by the
+    # raylet. However, make sure it is at least 1.
+    maximum_startup_concurrency = max(
+        1, min(psutil.cpu_count(), static_resources["CPU"]))
+
     # Format the resource argument in a form like 'CPU,1.0,GPU,0,Custom,3'.
     resource_argument = ",".join([
         "{},{}".format(resource_name, resource_value)
@@ -1035,6 +1040,7 @@ def start_raylet(redis_address,
         gcs_ip_address,
         gcs_port,
         str(num_workers),
+        str(maximum_startup_concurrency),
         resource_argument,
         start_worker_command,
         "",  # Worker command for Java, not needed for Python.