305eb74a86
[serve] Make HTTP proxy fault tolerant ( #7936 )
2020-04-09 17:07:22 -05:00
9bfc2c4b54
Moving Local Mode to C++ ( #7670 )
2020-04-01 15:50:57 -05:00
ee8c9ff732
Remove six and cloudpickle from setup.py. ( #7694 )
2020-03-23 11:42:05 -07:00
58dc70f90e
[minor] Remove get_global_worker(), RuntimeContext ( #7638 )
2020-03-20 15:45:29 -05:00
7b609ca211
Remove instances of 'raise Exception' ( #7523 )
2020-03-10 17:51:22 -07:00
fdb528514b
[core] Ref counting for actor handles ( #7434 )
...
* tmp
* Move Exit handler into CoreWorker, exit once owner's ref count goes to 0
* fix build
* Remove __ray_terminate__ and add test case for distributed ref counting
* lint
* Remove unused
* Fixes for detached actor, duplicate actor handles
* Remove unused
* Remove creation return ID
* Remove ObjectIDs from python, set references in CoreWorker
* Fix crash
* Fix memory crash
* Fix tests
* fix
* fixes
* fix tests
* fix java build
* fix build
* fix
* check status
* check status
2020-03-10 17:45:07 -07:00
93fe4b0b58
Change actor.__ray_kill__() to ray.kill(actor) ( #7360 )
2020-02-28 11:55:13 -06:00
d9027acaf2
Deprecate non-direct-call API ( #7336 )
2020-02-27 10:37:23 -08:00
5df801605e
Add ray.util package and move libraries from experimental ( #7100 )
2020-02-18 13:43:19 -08:00
a6b8bd47b0
[xlang] Cross language serialize ActorHandle ( #7134 )
2020-02-17 20:44:56 +08:00
0648bd28ef
[xlang] Cross language Python support ( #6709 )
2020-02-08 13:01:28 +08:00
396d7fafc8
UI improvement for asyncio ( #6905 )
2020-01-27 12:45:51 -08:00
c480d1d1e4
Treat static methods as class methods instead of instance methods in actors ( #6756 )
...
* Treat static methods as class methods rather than instance methods
* Add tests for static methods in actors
* Revert formatting changes
* Readd future imports
* Restructure static method check
* Documentation enhancements
* Fix linting issues
2020-01-15 19:38:41 -06:00
60d4d5e1aa
Remove future imports ( #6724 )
...
* Remove all __future__ imports from RLlib.
* Remove (object) again from tf_run_builder.py::TFRunBuilder.
* Fix 2xLINT warnings.
* Fix broken appo_policy import (must be appo_tf_policy)
* Remove future imports from all other ray files (not just RLlib).
* Remove future imports from all other ray files (not just RLlib).
* Remove future import blocks that contain `unicode_literals` as well.
Revert appo_tf_policy.py to appo_policy.py (belongs to another PR).
* Add two empty lines before Schedule class.
* Put back __future__ imports into determine_tests_to_run.py. Fails otherwise on a py2/print related error.
2020-01-09 00:15:48 -08:00
9fe90cdafc
Fix async actor recursion limitation ( #6672 )
...
* Do not start threadpool when using async
* Turn function_executor into a generator
* Add new test for high concurrency and bump the default
* Set direct call
2020-01-02 19:45:13 -06:00
39a3459886
Remove (object) from class declarations. ( #6658 )
2020-01-02 17:42:13 -08:00
46acb02aa4
Fix verbose shutdown error and test_env_with_subprocesses ( #6614 )
2019-12-26 22:43:39 -08:00
d3db9e9c1e
By default, reconstruction should only be enabled for actor creation. ( #6613 )
...
* wip
* fix
* fix
2019-12-26 19:57:50 -08:00
6b1a57542e
Add actor.__ray_kill__() to terminate actors immediately ( #6523 )
2019-12-23 23:12:57 -06:00
c57dcc82d1
Port actor creation to use direct calls ( #6375 )
2019-12-12 19:50:51 -08:00
5a5c94939f
[direct call] Retry failed tasks with delay ( #6453 )
...
* retry failed tasks with delay
* set to 0 for direct tests
2019-12-12 17:12:38 -08:00
aa8d5d2f6c
Rate limit asyncio actor ( #6242 )
2019-11-24 11:39:28 -08:00
29ba6bfc64
Basic Async Actor Call ( #6183 )
...
* Start trying to figure out where to put fibers
* Pass is_async flag from python to context
* Just running things in fiber works
* Yield implemented, need some debugging to make it work
* It worked!
* Remove debug prints
* Lint
* Revert the clang-format
* Remove unnecessary log
* Remove unncessary import
* Add attribution
* Address comment
* Add test
* Missed a merge conflict
* Make test pass and compile
* Address comment
* Rename async -> asyncio
* Move async test to py3 only
* Fix ignore path
2019-11-21 11:56:46 -08:00
0a3623ded6
Fix memory store wait ( #6152 )
2019-11-14 10:17:30 -08:00
8485304e83
Support concurrent Actor calls in Ray ( #6053 )
2019-11-04 01:14:35 -08:00
fbad6f543b
Try fixing actor handle destruction on py2 ( #6076 )
2019-11-03 22:46:40 -08:00
fb34928a2a
[minor] Perf optimizations for direct actor task submission ( #6044 )
...
* merge optimizations
* fix
* fix memory err
* optimize
* fix tests
* fix serialization of method handles
* document weakref
* fix check
* bazel format
* disable on 2
2019-11-01 14:41:14 -07:00
7f5b3502da
Implement Detached Actor ( #6036 )
...
* Arg propagation works
* Implement persistent actor
* Add doc
* Initialize is_persistent_
* Rename persistent->detached
* Address comment
* Make test passes
* Address comment
* Python2 compatiblity
* Fix naming, py2
* Lint
2019-11-01 10:28:23 -07:00
b89cac976a
Basic direct actor call support in Python ( #5991 )
2019-10-28 22:09:04 -07:00
09d05bb3fa
Reduce actor submission python overhead ( #5949 )
2019-10-23 00:11:32 -07:00
26a724c5e6
[core] Support kwargs and positionals in Ray remote calls ( #5606 )
2019-10-20 22:40:54 -07:00
697f765efc
Refactor CoreWorker to remove TaskInterface ( #5924 )
...
* Remove TaskInterface
* Remove Status return value
* Remove CActorHandle, some return values, TaskSubmitter
* lint
* doc
* doc
* fix build
* lint
* Return Status, guarded by annotation, fail tasks for RECONSTRUCTING actors
* fix
* move annotation
* revert
* Fix core worker test
* nits
2019-10-18 00:03:57 -04:00
3ac8592dcf
Remove actor handle IDs ( #5889 )
...
* Remove actor handle ID from main ActorHandle constructor
* Set the actor caller ID when calling submit task instead of in the actor handle
* Remove ActorHandle::Fork, remove actor handle ID from protobuf
* Make inner actor handle const, remove new_actor_handles
* Move caller ID into the common task spec, start refactoring raylet
* Some fixes for forking actor handles
* Store ActorHandle state in CoreWorker, only expose actor ID to Python
* Remove some unused fields
* lint
* doc
* fix merge
* Remove ActorHandleID from python/cpp
* doc
* Fix core worker test
* Move actor table subscription to CoreWorker, reset actor handles on actor failure
* lint
* Remove GCS client from direct actor
* fix tests
* Fix
* Fix tests for raylet codepath
* Fix local mode
* Fix multithreaded test
* Fix AsyncSubscribe issue...
* doc
* fix serve
* Revert bazel
2019-10-17 12:36:34 -04:00
42dd0fae96
Fix actor ID collision in local mode ( #5863 )
...
* Fixed local mode actor id
* Update python/ray/actor.py
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com >
* Added hyphen to match comments
* Added tests to test_local_mode
* Helloworld
* Better test naming
* lint
2019-10-08 13:07:42 -07:00
785670bc18
Fix class attributes and methods for actor classes ( #5802 )
2019-10-07 23:56:07 -07:00
08e4e3a153
[core worker] Submit Python actor tasks through core worker ( #5750 )
...
* Submit actor tasks through core worker
* Fix java
* add comment
* Remove task builder
* Check negative
* Increase -> Increment
* pass by reference
* fix signal
* Clean up c++ actor handle
* more cleanup
* Clean up headers
* Fix unique_ptr construction
* Fix java
* Move profiling to c++
* dedup
* fix error
* comments
* fix java
* Fix tests
* wait for actor to exit
* Start after constructor
* ignore java build
* fix comment
* always init logging
* Fix logging
* fix logging issue
* shared_ptr for profiler
* DEBUG -> WARNING
* fix killed_ init
* Fix flaky checkpointing tests
* -v flag for tune tests
* Fix checkpoint test logic
* Fix exception matching
* timeout exception
* Fix test exception info
* Fix import
* fix build
* Fix test
* shared_ptr
2019-10-07 15:42:19 -07:00
07c4c6367a
[core worker] Python core worker object interface ( #5272 )
2019-09-12 23:07:46 -07:00
4ed6ee0b1e
Better error message for actor class inheritance ( #5598 )
2019-09-03 10:42:27 -07:00
e2e30ca507
Ray, Tune, and RLlib support for memory, object_store_memory options ( #5226 )
2019-08-21 23:01:10 -07:00
d372f24e3c
[ID Refactor] Refactor ActorID, TaskID and ObjectID ( #5286 )
...
* Refactor ActorID, TaskID on the Java side.
Left a TODO comment
WIP for ObjectID
ADD test
Fix
Add java part
Fix Java test
Fix
Refine test.
Enable test in CI
* Extra a helper function.
* Resolve TODOs
* Fix Python CI
* Fix Java lint
* Update .travis.yml
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Address some comments.
Address some comments.
Add id_specification.rst
Reanme id_specification.rst to id_specification.md
typo
Address zhijun's comments.
Fix test
Address comments.
Fix lint
Address comments
* Fix test
* Address comments.
* Fix build error
* Update src/ray/design_docs/id_specification.md
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Update src/ray/design_docs/id_specification.md
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Update src/ray/design_docs/id_specification.md
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Update src/ray/design_docs/id_specification.md
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Update src/ray/design_docs/id_specification.md
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Address comments
* Update src/ray/common/id.h
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Update src/ray/common/id.h
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Update src/ray/common/id.h
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Update src/ray/design_docs/id_specification.md
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update src/ray/design_docs/id_specification.md
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Address comments.
* Address comments.
* Address comments.
* Update C++ part to make sure task id is generated determantic
* WIP
* Fix core worker
* Fix Java part
* Fix comments.
* Add Python side
* Fix python
* Address comments
* Fix linting
* Fix
* Fix C++ linting
* Add JobId() method to TaskID
* Fix linting
* Update src/ray/common/id.h
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update java/api/src/main/java/org/ray/api/id/TaskId.java
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update java/api/src/main/java/org/ray/api/id/TaskId.java
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update java/api/src/main/java/org/ray/api/id/ActorId.java
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Address comments
* Add DriverTaskId embeding job id
* Fix tests
* Add python dor_fake_driver_id
* Address comments and fix linting
* Fix CI
2019-08-07 11:04:51 +08:00
e5be5fd46d
Remove dependencies from TaskExecutionSpecification ( #5166 )
2019-07-15 18:15:21 -07:00
f2293243cc
[ID Refactor] Shorten the length of JobID to 4 bytes ( #5110 )
...
* WIP
* Fix
* Add jobid test
* Fix
* Add python part
* Fix
* Fix tes
* Remove TODOs
* Fix C++ tests
* Lint
* Fix
* Fix exporting functions in multiple ray.init
* Fix java test
* Fix lint
* Fix linting
* Address comments.
* FIx
* Address and fix linting
* Refine and fix
* Fix
* address
* Address comments.
* Fix linting
* Fix
* Address
* Address comments.
* Address
* Address
* Fix
* Fix
* Fix
* Fix lint
* Fix
* Fix linting
* Address comments.
* Fix linting
* Address comments.
* Fix linting
* address comments.
* Fix
2019-07-11 14:25:16 +08:00
8f53364097
Improve local_mode ( #5060 )
2019-07-07 17:10:50 -07:00
62e4b591e3
[ID Refactor] Rename DriverID to JobID ( #5004 )
...
* WIP
WIP
WIP
Rename Driver -> Job
Fix complition
Fix
Rename in Java
In py
WIP
Fix
WIP
Fix
Fix test
Fix
Fix C++ linting
Fix
* Update java/runtime/src/main/java/org/ray/runtime/config/RayConfig.java
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Update src/ray/core_worker/core_worker.cc
Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu >
* Address comments
* Fix
* Fix CI
* Fix cpp linting
* Fix py lint
* FIx
* Address comments and fix
* Address comments
* Address
* Fix import_threading
2019-06-28 00:44:51 +08:00
6703519144
Move global state API out of global_state object. ( #4857 )
2019-05-26 11:27:53 -07:00
49fe894e22
Export remote functions when first used and also fix bug in which rem… ( #4844 )
...
* Export remote functions when first used and also fix bug in which remote functions and actor classes are not exported from workers during subsequent ray sessions.
* Documentation update
* Fix tests.
* Fix grammar
2019-05-24 13:44:39 -07:00
2015085192
Fix bug in which actor classes are not exported multiple times. ( #4838 )
2019-05-23 09:22:46 -07:00
1a39fee9c6
Refactor ID Serial 1: Separate ObjectID and TaskID from UniqueID ( #4776 )
...
* Enable BaseId.
* Change TaskID and make python test pass
* Remove unnecessary functions and fix test failure and change TaskID to
16 bytes.
* Java code change draft
* Refine
* Lint
* Update java/api/src/main/java/org/ray/api/id/TaskId.java
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update java/api/src/main/java/org/ray/api/id/BaseId.java
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update java/api/src/main/java/org/ray/api/id/BaseId.java
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update java/api/src/main/java/org/ray/api/id/ObjectId.java
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Address comment
* Lint
* Fix SINGLE_PROCESS
* Fix comments
* Refine code
* Refine test
* Resolve conflict
2019-05-22 14:46:30 +08:00
d81e71e297
Enable actor methods to be decorated on the caller side also and get postprocessors. ( #4732 )
...
* Allow decorating ray actor methods.
* Add test.
* Add get postprocessors.
* Improve documentation.
* Make it work for remote functions.
* Temporary fix.
2019-05-04 11:53:47 -07:00
4eade036a0
Separate thread locks for worker and function manager. ( #4499 )
...
* Separate lock for function manager and worker
* Lint
* Add test case
* Remove print in remote function.
* Remove test and add ray.exit_actor
* Update python/ray/worker.py
Co-Authored-By: guoyuhong <guoyuhong1985@outlook.com >
* Move exit_actor from worker.py to actor.py
* Update actor.py
* Update actor.py
2019-04-29 14:55:37 +08:00
Edward Oakes