Commit Graph

276 Commits

Author SHA1 Message Date
Philipp Moritz 39ce0eadbe Ray PDB support (#11739) 2020-11-03 09:49:23 -08:00
Stephanie Wang ada58abcd9 [Object spilling] Update object directory and reload spilled objects automatically (#11021)
* Fix pytest...

* Release objects that have been spilled

* GCS object table interface refactor

* Add spilled URL to object location info

* refactor to include spilled URL in notifications

* improve tests

* Add spilled URL to object directory results

* Remove force restore call

* Merge spilled URL and location

* fix

* CI

* build

* osx

* Fix multitenancy issues

* Skip windows tests
2020-10-02 15:52:42 -07:00
DK.Pino db7097fb1f [Refactor] Rename ClientId to NodeId (#10992)
* rename ClientId to NodeId

* format lint

* format lint

* fix conflicts

* rename new ClientId to NodeId

* update lint

* make same version of clang-format with travis ci
2020-09-27 10:24:21 -07:00
SangBin Cho 109481afd9 [Metric] custom metrics refinement (#10861)
* In progress

* In Progress.

* Addressed code review.

* Add unit tests.

* Add a simple doc.

* Fixed test failure.

* Fix all test failures from serve.

* Addressed code review.
2020-09-25 09:10:28 -07:00
Simon Mo fdd3acd492 Promote ray.experimental.queue to ray.util (#10624) 2020-09-08 12:56:53 -07:00
Eric Liang 8ee7c182f5 [1.0] move placement groups from experimental to util. Note they are still undocumented. (#10554)
* move files

* Update __init__.py

* remove

* Update __init__.py
2020-09-04 19:01:24 -07:00
Eric Liang da83bbd764 [1.0] Move dask scheduler from experimental to util (#10553)
* move dask

* fix dask
2020-09-04 12:16:32 -07:00
Clark Zinzow 7068c63dd8 Set Ray task name to Dask key for Dask tasks. (#10547) 2020-09-03 15:37:55 -07:00
Eric Liang 2a204260a8 [api] Second round of 1.0 API changes: exceptions, num_return_vals (#10377) 2020-08-28 19:57:02 -07:00
Eric Liang 519354a39a [api] Initial API deprecations for Ray 1.0 (#10325) 2020-08-28 15:03:50 -07:00
SangBin Cho 3b3ca96a4e [Placement Group] Wait (#10259)
* Initial progress done.

* Fix wrong test.

* Improve tests.

* Update code.

* Addressed code review and merge conflict.

* Addressed code review.
2020-08-24 20:14:48 -07:00
fangfengbin 36c6c4b298 [Placement group] Check if placement group bundle index is valid (#10194)
* add part code

* rebase master

* add java testcase

* fix review comments

* fix lint error

* rebase master

* fix lint error

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-08-21 11:04:56 -07:00
fangfengbin a462ae2747 [Placement Group]Add strict spread strategy (#10174)
* support STRICT_SPREAD strategy

* fix review comments

* rebase master

* fix lint error

* fix lint error

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-08-20 10:18:58 -07:00
architkulkarni de46464aa3 [Experimental] Queue: replace polling with async actor (#10120) 2020-08-19 11:55:42 -05:00
SangBin Cho 263df6163c [Placement Group] Placement group remove api part 1 (#10063)
* Added basic rpc calls.

* fix issues.

* Fix the gcs server not getting request issue.

* In Progress.

* Basic logic done. Tests are required.

* In progress.

* In progress in refactoring context.

* Revert "In progress in refactoring context."

This reverts commit 38236256cf1306c60dd203e75d45ceb4509c8106.

* Working now.

* Python test works.

* Lint.

* Addressed code review.

* Addressed code review.

* Lint.

* Added unit tests.

* Done, but one of unit tests fail

* Addressed code review.

* Addressed the last code review.

* Fix the wrong test case.
2020-08-18 12:44:00 -07:00
SangBin Cho 053188dfbe [Placement Group] Support Placement Group state table. (#10090)
* Done.

* Addressed code review.

* Linting.

* Fix lint.

* Fix lint.

* Fix a test.

* Lint.

* Add a lint sleep to test.

* Fix the lint issue.

* Fixed doc build error.
2020-08-17 09:24:50 -07:00
fangfengbin edd783bc32 [Placement Group]Add soft pack strategy (#10099) 2020-08-17 12:01:34 +08:00
Siyuan (Ryans) Zhuang 17ca1d8ff4 [Core] Object spilling prototype (#9818) 2020-08-14 15:39:10 -07:00
Clark Zinzow 6fded582ff [Dask] Dask-Ray scheduler MVP. (#9857) 2020-08-04 11:45:25 -07:00
sanderland 323bc23c21 Fix copy-paste error in queue.empty (#9757) 2020-08-03 14:14:18 -05:00
Eric Liang b73080c85f Allow tasks to be used with placement groups (#9738) 2020-07-31 10:51:37 -07:00
Alisa 51e12ee97c Python api of placement group (#9243) 2020-07-27 14:57:05 -07:00
Lingxuan Zuo ce3f542739 [Metric] new cython interface for python worker metric (#9469) 2020-07-19 10:43:21 +08:00
Hao Chen d49dadf891 Change Python's ObjectID to ObjectRef (#9353) 2020-07-10 17:49:04 +08:00
Ian Rodney a1e14380ce [core] Switch Async Callback to C++ [WIP] (#9228)
Co-authored-by: simon-mo <simon.mo@hey.com>
2020-07-07 09:47:25 -07:00
Sven Mika c4ccbfdfa9 [RLlib] tf-eager support for ES and ARS (tf2.x preparation). (#9207) 2020-07-02 13:03:10 +02:00
Sven Mika 43043ee4d5 [RLlib] Tf2x preparation; part 2 (upgrading try_import_tf()). (#9136)
* WIP.

* Fixes.

* LINT.

* WIP.

* WIP.

* Fixes.

* Fixes.

* Fixes.

* Fixes.

* WIP.

* Fixes.

* Test

* Fix.

* Fixes and LINT.

* Fixes and LINT.

* LINT.
2020-06-30 10:13:20 +02:00
Sven Mika 4fd8977eaf [RLlib] Minor cleanup in preparation to tf2.x support. (#9130)
* WIP.

* Fixes.

* LINT.

* Fixes.

* Fixes and LINT.

* WIP.
2020-06-25 19:01:32 +02:00
Edward Oakes c9010eb8ad [serve] Add serve.shutdown() (#8766) 2020-06-23 13:42:03 -05:00
Sven Mika 25c0974543 [RLlib] Issue 8412 (Adam vars not stored in ModelV2). (#8480) 2020-06-05 21:07:02 +02:00
internetcoffeephone a73c488c74 Change tf_utils.py get_weights to evaluate all tensors at once rather than calling tensor.eval per-tensor. (#8491) 2020-05-18 22:06:03 -07:00
Simon Mo 13c14eac07 [Asyncio] Remove async init legacy code (#8177)
* [Asyncio] Remove async init legacy code

* Fix places that call async_init
2020-04-25 09:32:38 -07:00
ijrsvt e03f687b84 Cleaning up remaining Local Mode Code (#7865) 2020-04-03 19:54:15 -05:00
Edward Oakes 58dc70f90e [minor] Remove get_global_worker(), RuntimeContext (#7638) 2020-03-20 15:45:29 -05:00
ijrsvt 46953c53b1 Cleanup Plasma Async Callback (#7452) 2020-03-16 10:12:44 -07:00
Edward Oakes 0c254295b0 Remove experimental.signal API (#7477)
* Remove experimental.signal API

* fix test
2020-03-09 16:03:36 -07:00
Edward Oakes b4e2d5317e Remove experimental.NoReturn (#7475) 2020-03-09 11:09:36 -07:00
Edward Oakes 0abcca258f Add entries to in-memory store on Put() (#7085) 2020-03-04 10:17:27 -08:00
ijrsvt fb76092d75 Re-route asyncio plasma code path through raylet instead of direct plasma connection (#7234) 2020-03-03 15:43:46 -05:00
ijrsvt 584645cc7d Fix Experimental Async API (#7391) 2020-03-02 22:24:20 -06:00
Edward Oakes d9027acaf2 Deprecate non-direct-call API (#7336) 2020-02-27 10:37:23 -08:00
Simon Mo b804d40c04 Stop vendoring pyarrow (#7233) 2020-02-19 19:01:26 -08:00
Simon Mo 7bef7031c2 Revert "Revert "Revert "Removing Pyarrow dependency (#7146)" (#7209) (#7214)" (#7232) 2020-02-19 13:35:29 -08:00
Simon Mo e8941b1b79 Revert "Revert "Removing Pyarrow dependency (#7146)" (#7209) (#7214) 2020-02-19 10:08:52 -08:00
Eric Liang 0aa9373d62 Revert "Removing Pyarrow dependency (#7146)" (#7209)
This reverts commit 2116fd3bca.
2020-02-18 14:12:06 -08:00
Eric Liang 5df801605e Add ray.util package and move libraries from experimental (#7100) 2020-02-18 13:43:19 -08:00
ijrsvt 2116fd3bca Removing Pyarrow dependency (#7146) 2020-02-17 18:00:13 -08:00
Alind Khare c6d768be14 [Serve] Added support for no http route services (#7010) 2020-02-17 11:31:30 -08:00
Richard Liaw 94e2fcea2e [sgd] fp16 (apex) and scheduler support + move examples page (#7061)
* Init fp16

* fp16 and schedulers

* scheduler linking and fp16

* to fp16

* loss scaling and documentation

* more documentation

* add tests, refactor config

* moredocs

* more docs

* fix logo, add test mode, add fp16 flag

* fix tests

* fix scheduler

* fix apex

* improve safety

* fix tests

* fix tests

* remove pin memory default

* rm

* fix

* Update doc/examples/doc_code/raysgd_torch_signatures.py

* fix

* migrate changes from other PR

* ok thanks

* pass

* signatures

* lint'

* Update python/ray/experimental/sgd/pytorch/utils.py

* Apply suggestions from code review

Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>

* should address most comments

* comments

* fix this ci

* fix tests'

* testmode

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
2020-02-16 19:04:08 -08:00
Edward Oakes dc5a27dac0 Move ray.experimental.multiprocessing to ray.util.multiprocessing (#7149) 2020-02-14 16:17:05 -08:00