Commit Graph

2556 Commits

Author SHA1 Message Date
Stephanie Wang 0389735d7a [core] Pass owner address from the workers to the raylet (#9299)
* Add intended worker ID to GetObjectStatus, tests

* Remove TaskID owner_id

* lint

* Add owner address to task args

* Make TaskArg a virtual class, remove multi args

* Set owner address for task args

* merge

* Fix tests

* Add ObjectRefs to task dependency manager, pass from task spec args

* tmp

* tmp

* Fix

* Add ownership info for task arguments

* Convert WaitForDirectActorCallArgs

* lint

* build

* update

* build

* java

* Move code

* build

* Revert "Fix Google log directory again (#9063)"

This reverts commit 275da2e400.

* Fix free

* fix tests

* Fix tests

* build

* build

* fix

* Change assertion to warning to fix java
2020-07-09 14:35:54 -07:00
fangfengbin 62fe403a57 Fix the bug that PollOwnerForActorOutOfScope's reply may be missing (#9330) 2020-07-09 17:33:52 +08:00
Kai Yang fa1eadd61f fix java worker log dir (#9352) 2020-07-09 10:11:03 +08:00
SangBin Cho 9f8ff2e3b1 [Core] GCS Actor Management Race Condition (#9215)
* GCS Actor management on by default.

* Fix travis config.

* Change condition.

* Finish the initial race condition fix.

* Lint.

* Refine the codebase.

* Finish the initial version

* Improve logic.

* Remove unnecessary log messages.

* Address code review.

* Add tests

* Revert the second race condition that doesn't happen anymore, handle some edge cases. add tests.

* Address the second race condition found.

* Addressed code review.

* Addressed code review.

* Run a new unit test only when gcs actor management is on.
2020-07-08 10:56:52 -07:00
Ian Rodney 9172f8c3a6 [core] Store Internal Config in GCS (#8921) 2020-07-08 11:22:08 -05:00
Sven Mika 4da0e542d5 [RLlib] DDPG and SAC eager support (preparation for tf2.x) (#9204) 2020-07-08 16:12:20 +02:00
Tao Wang f395e48031 [GCS] Add integration test for actor info cleaning (#8900) 2020-07-08 10:25:08 +08:00
Max Fitton ca54da66b6 Worker <> GPU Mapping Information in Dashboard (#9163) 2020-07-07 18:58:22 -05:00
yncxcw 4ba4110dec [Core] Make worker_register_timeout_seconds configurable (#9221) 2020-07-07 18:27:15 -05:00
SangBin Cho 8f19f1eafb [Core] Actor handle refactoring (#8895)
* Marking needed changes.

* Resolve basic dependencies.

* In progress.

* linting.

* In progress 2.

* Linting.

* Refactor done. Cleanup needed.

* Linting.

* Recover kill actor in core worker because it is used inside raylet

* Cleanup.

* Use unique pointer instead. Unit tests are broken now.

* Fix the upstream change.

* Addressed code review 1.

* Lint.

* Addressed code review 2.

* Fix weird github history.

* Lint.

* Linting using clang 7.0.

* Use a better check message.

* Revert cpp stuff.

* Fix weird linting errors.

* Manuall fix all lint issues.

* Update a newline.

* Refactor some interface.

* Addressed all code review.

* Addressed code review
2020-07-07 11:11:41 -07:00
Ian Rodney f69d8b951e [core] Disable flaky cancellation test (#9325) 2020-07-07 09:52:35 -07:00
Ian Rodney a1e14380ce [core] Switch Async Callback to C++ [WIP] (#9228)
Co-authored-by: simon-mo <simon.mo@hey.com>
2020-07-07 09:47:25 -07:00
Ian Rodney 079c1eaa5c [autoscaler] Require keyword arguments (#9256) 2020-07-06 22:12:59 -07:00
Stephanie Wang b42d6a1ddc [core] Refactor task arguments and attach owner address (#9152)
* Add intended worker ID to GetObjectStatus, tests

* Remove TaskID owner_id

* lint

* Add owner address to task args

* Make TaskArg a virtual class, remove multi args

* Set owner address for task args

* merge

* Fix tests

* Fix

* build

* update

* build

* java

* Move code

* build

* Revert "Fix Google log directory again (#9063)"

This reverts commit 275da2e400.

* Fix free

* x

* build

* Fix java

* Revert "Revert "Fix Google log directory again (#9063)""

This reverts commit 4a326fcb148ca09a35bc7de11d89df10edbb56e7.

* lint
2020-07-06 21:25:14 -07:00
Ian Rodney 6fecd3cfce [autoscaler] Run initialization_commands without a persistent connection (#9020)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
2020-07-06 16:34:59 -07:00
Richard Liaw 139d21e068 [tune] Docs for tune-sklearn (#9129)
Co-authored-by: krfricke <krfricke@users.noreply.github.com>
2020-07-06 15:35:10 -07:00
SangBin Cho 557da7044f Fix flaky test that says ray.init is called twice. (#9234) 2020-07-06 15:19:00 -07:00
Ian Rodney c79a49488c [Doc] Fixing Wheel Build Command (#8789) 2020-07-06 11:17:59 -07:00
ChenZhilei 6f3d993681 GCS server use worker table to handle RegisterWorker instead of redis accessor (#9168) 2020-07-06 10:37:25 +08:00
Richard Liaw b71c912da7 [tune] Fix up examples (#9201) 2020-07-05 01:16:20 -07:00
Tao Wang f7ac495a68 [Core] Use map instead of list to represent resources in heartbeat message (#9294) 2020-07-05 10:59:25 +08:00
mehrdadn 28d18ec836 [windows] Improve GPU detection (#9300)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-04 13:20:23 -07:00
David Fidalgo c0ba337fe0 [tune] Add np.bool8 and np.int to allowed HPARAMS types (#9297) 2020-07-03 18:34:45 -07:00
mehrdadn 7135cb2aec Fix .exe file extensions (#9197)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-02 15:29:34 -05:00
Simon Mo a25472c657 [Serve] Fix prometheus exporter label override (#9227) 2020-07-02 11:40:14 -07:00
Ian Rodney f1173d55e0 [autoscaler] Fix cluster_shutdown using wrong args (#9252) 2020-07-02 09:34:50 -07:00
SangBin Cho bfb8a28a3c Revert "Support accessing underlying attributes in RayTaskErrors (#9210)" (#9263)
This reverts commit 21cad5250a.
2020-07-02 08:35:10 -05:00
Sven Mika c4ccbfdfa9 [RLlib] tf-eager support for ES and ARS (tf2.x preparation). (#9207) 2020-07-02 13:03:10 +02:00
fangfengbin 8fcfcc4100 GCS server error handling for actor creation (#8899) 2020-07-02 16:27:32 +08:00
Eric Liang 82a5ed3fb5 [autoscaler] Instance type assert fix for non-AWS node providers (#9223) 2020-07-01 17:54:20 -07:00
Andrija Djurisic e0d8b58969 Fix Azure example-gpu.yml (#9242)
`imageSku` should be string ... otherwise this is the error it shows up:
```
msrestazure.azure_exceptions.CloudError: Azure Error: InvalidTemplate
Message: Deployment template validation failed: 'Template parameter JToken type is not valid. Expected 'String, Uri'. Actual 'Integer'. Please see https://aka.ms/resource-manager-parameter-files for usage details.'.
Additional Information:
	Type: TemplateViolation
	Info: {
	    "lineNumber": 1,
	    "linePosition": 673,
	    "path": "properties.template.parameters.imageSku"
	}

```
2020-07-01 12:02:34 -07:00
Richard Liaw d35f0e40d0 [tune] Use public methods for trainable (#9184) 2020-07-01 11:00:00 -07:00
Edward Oakes 21cad5250a Support accessing underlying attributes in RayTaskErrors (#9210) 2020-07-01 10:49:38 -05:00
Sven Mika b4c0b942fe [RLlib] Remove requirement for dataclasses in rllib (not supported in py3.5) (#9237) 2020-07-01 17:31:44 +02:00
ChenZhilei c11855728a Remove raylet monitor after use GCS service (#9179) 2020-07-01 20:01:52 +08:00
Siyuan (Ryans) Zhuang a1dfcfc893 Add test for Ray outputs (#9231) 2020-06-30 23:54:54 -07:00
Ian Rodney 4ce4be5c54 Show actual command in output (#9224) 2020-06-30 16:00:28 -07:00
Simon Mo be647b69ab [Serve] Performance: Use uvloop when possible (#9216) 2020-06-30 15:59:13 -07:00
Edward Oakes 681bae709a [serve] Add backend config options to docstrings (#9193) 2020-06-30 10:53:17 -05:00
Sven Mika 43043ee4d5 [RLlib] Tf2x preparation; part 2 (upgrading try_import_tf()). (#9136)
* WIP.

* Fixes.

* LINT.

* WIP.

* WIP.

* Fixes.

* Fixes.

* Fixes.

* Fixes.

* WIP.

* Fixes.

* Test

* Fix.

* Fixes and LINT.

* Fixes and LINT.

* LINT.
2020-06-30 10:13:20 +02:00
Ian Rodney fb074da7c3 [core] Return all GPUs for Local Mode (#9108) 2020-06-30 00:55:09 -07:00
mehrdadn 882f60012f Fix WSL patch (it doesn't fall under "win32") (#9195) 2020-06-29 21:00:00 -05:00
krfricke e0b6984dce [tune] pytorch lightning template and walkthrough (#9151)
Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-06-29 16:52:07 -07:00
SangBin Cho 7af6c69672 [Test] Cluster util fix (#8929) 2020-06-29 14:15:41 -05:00
Edward Oakes 87e47c4a53 Fix serve dependencies (#9192) 2020-06-29 12:44:04 -05:00
mehrdadn bc140b69cc Patch WSL psutil bug (#6808) 2020-06-29 10:55:43 -05:00
mehrdadn f4e5561ef7 Fix build error (#9185)
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-29 14:51:01 +08:00
mehrdadn 8f72a9fd1a Fix more Windows tests (#9173) 2020-06-28 17:13:15 +08:00
mehrdadn 898e472425 Make test_utils.py use pipes to avoid file access conflicts on Windows (#9072)
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-27 22:33:45 +02:00
Richard Liaw 625b66bde7 [docs] Show commands on documentation (#9165) 2020-06-27 12:56:56 -07:00