Commit Graph

196 Commits

Author SHA1 Message Date
Qing Wang ad90693ca8 [Java] Enhancement single process mode (#6795)
* enhancement

* Add ut

* Update java/runtime/src/main/java/org/ray/runtime/task/LocalModeTaskSubmitter.java

Co-Authored-By: Kai Yang <kfstorm@outlook.com>

* Update java/test/src/main/java/org/ray/api/test/RunModeTest.java

Co-Authored-By: Kai Yang <kfstorm@outlook.com>

* Address comments

* Use ExecutorSerivce to replace raw thread

Co-authored-by: Kai Yang <kfstorm@outlook.com>
2020-01-15 21:38:53 +08:00
Kai Yang cd5fc81bdd Show the exit code if RunManager failed to start a process (#6797) 2020-01-15 19:31:56 +08:00
Kai Yang ddd4c42fe5 [Java] Add killActor API in Java (#6728)
* Add killActor API in Java

* fix javadoc

* update test case

* Address comments
2020-01-14 17:12:00 +08:00
chaokunyang 36b2634479 [Java] Remove singleton exceptions (#6765) 2020-01-13 10:28:06 +08:00
chaokunyang 723fe86882 [Java] Fix building Java with maven (#6764)
* lint

* gen_maven_deps for ray_java_pkg
2020-01-11 14:26:27 +08:00
chaokunyang 4097d076d4 Package ray java jars into wheels (#6600) 2020-01-10 11:41:00 +08:00
Sven 60d4d5e1aa Remove future imports (#6724)
* Remove all __future__ imports from RLlib.

* Remove (object) again from tf_run_builder.py::TFRunBuilder.

* Fix 2xLINT warnings.

* Fix broken appo_policy import (must be appo_tf_policy)

* Remove future imports from all other ray files (not just RLlib).

* Remove future imports from all other ray files (not just RLlib).

* Remove future import blocks that contain `unicode_literals` as well.
Revert appo_tf_policy.py to appo_policy.py (belongs to another PR).

* Add two empty lines before Schedule class.

* Put back __future__ imports into determine_tests_to_run.py. Fails otherwise on a py2/print related error.
2020-01-09 00:15:48 -08:00
Robert Nishihara 5e43b25e8c Document fault tolerance behavior. (#6698) 2020-01-06 22:34:06 -08:00
Zhijun Fu d2bba596ab Fix actor reconstruction with direct call (#6570) 2019-12-26 10:59:50 +08:00
Chaokun Yang 7bbfa85c66 [Streaming] Streaming data transfer java (#6474) 2019-12-22 10:56:05 +08:00
Chaokun Yang eca4cc7c00 Fix plasma_store_server path (#6499) 2019-12-20 14:45:51 +08:00
Kai Yang af9f76359a Fix crash caused by unit test testWaitAndCrash (#6549) 2019-12-20 11:06:53 +08:00
Kai Yang 3bb680a719 [Java] Add getAsyncContext and setAsyncContext API (#6439)
* Add getAsyncContext and setAsyncContext API

* address comment

* fix bug

* Add test case
2019-12-19 18:08:58 +08:00
Kai Yang 03c37c7825 [Java] Add the default redis password for Java (#6528) 2019-12-19 10:53:53 +08:00
Kai Yang 7e6defca72 fix ClassLoaderTest.java (#6502) 2019-12-18 14:48:25 +08:00
Kai Yang b7d5c8f220 [Java] Fix multiple FunctionManagers creating multiple ClassLoader s (#6434) 2019-12-16 14:04:44 +08:00
Kai Yang c2499c802f disable actor checkpointing and reconstruction test in direct call mode (#6490) 2019-12-15 17:54:31 +08:00
Kai Yang cd250ba0bc [Java] ID length fix (#6454) 2019-12-15 16:01:05 +08:00
Hao Chen 5cc3e1341a [Java] Cache result in RayObjectImpl (#6414) 2019-12-11 11:26:01 +08:00
Kai Yang eb912b68b1 [Java] Fix instanceof RayPyActor (#6377) 2019-12-07 16:28:29 +08:00
Kai Yang 7e9fddf3ed [Java] Add java exception check in JNI (#6378) 2019-12-07 16:25:17 +08:00
Zhijun Fu b88b8202cc fix java build failure (#6062) 2019-12-06 14:38:43 +08:00
Kai Yang d51583dbd6 Add test listener to show the test progress of java UT (#6341) 2019-12-03 16:34:07 +08:00
ashione a1744f67fe Add hostname to nodeinfo(#6156) 2019-11-19 15:03:46 +08:00
Philipp Moritz ccbcc4bafa Use GRCP and Bazel 1.0 (#6002) 2019-11-08 15:58:28 -08:00
Qing Wang 4636fc2b78 Fix java ci (#5964) 2019-10-30 14:50:53 -07:00
Kai Yang c580955840 [Java] Fix some potential bugs about Ray.shutdown() (#5693) 2019-09-24 10:44:17 +08:00
Kai Yang 1b880191b0 Replace NotImplementedException with UnsupportedOperationException (#5694) 2019-09-12 00:40:26 -07:00
Kai Yang ed761900f6 [Java] Support direct actor call in Java worker (#5504) 2019-09-09 14:29:20 +08:00
Kai Yang d8f5804690 Support metadata for passing by value task arguments (#5527) 2019-09-08 11:07:48 +08:00
Kai Yang 732336fc4f [Java] Support multiple workers in Java worker process (#5505) 2019-09-07 22:52:05 +08:00
Kai Yang fadfa5f30b [Java] ObjectID::fromRandom sets proper flags (#5548) 2019-08-28 11:31:06 +08:00
Hao Chen fab5ae64c4 [Java] Automatically clean up temp files. (#5507)
* Remove unused code

* Auto clean up temp files

* refine

* fix

* print warning

* fix
2019-08-24 17:20:27 +08:00
Kai Yang 53fd66f6d6 [Java] Destroy native core worker before killing ray processes (#5516) 2019-08-24 11:10:02 +08:00
Hao Chen 239c177fe8 [Java] Support calling functions returning void (#5494) 2019-08-23 21:10:15 +08:00
Kai Yang 7812dd5636 [Java] Fix getCurrentActorId in multi-threading scenario. (#5506) 2019-08-23 17:56:10 +08:00
Philipp Moritz 599cc2be60 Revert raylet to worker GRPC communication back to asio (#5450) 2019-08-17 19:11:32 -07:00
Kai Yang b1aae0e398 [Java worker] Migrate task execution and submission on top of core worker (#5370) 2019-08-16 13:52:13 +08:00
Qing Wang d372f24e3c [ID Refactor] Refactor ActorID, TaskID and ObjectID (#5286)
* Refactor ActorID, TaskID on the Java side.

Left a TODO comment

WIP for ObjectID

ADD test

Fix

Add java part

Fix Java test

Fix

Refine test.

Enable test in CI

* Extra a helper function.

* Resolve TODOs

* Fix Python CI

* Fix Java lint

* Update .travis.yml

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Address some comments.

Address some comments.

Add id_specification.rst

Reanme id_specification.rst to id_specification.md

typo

Address zhijun's comments.

Fix test

Address comments.

Fix lint

Address comments

* Fix test

* Address comments.

* Fix build error

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Address comments

* Update src/ray/common/id.h

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/common/id.h

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/common/id.h

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Address comments.

* Address comments.

* Address comments.

* Update C++ part to make sure task id is generated determantic

* WIP

* Fix core worker

* Fix Java part

* Fix comments.

* Add Python side

* Fix python

* Address comments

* Fix linting

* Fix

* Fix C++ linting

* Add JobId() method to TaskID

* Fix linting

* Update src/ray/common/id.h

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/api/src/main/java/org/ray/api/id/TaskId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/api/src/main/java/org/ray/api/id/TaskId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/api/src/main/java/org/ray/api/id/ActorId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Address comments

* Add DriverTaskId embeding job id

* Fix tests

* Add python dor_fake_driver_id

* Address comments and fix linting

* Fix CI
2019-08-07 11:04:51 +08:00
Kai Yang 384cbfb211 Fix duplicated timeout logic in AbstractRayRuntime.get() (#5338) 2019-08-06 13:36:49 +08:00
micafan b3bcf59148 Rename ClientTableData to GcsNodeInfo (#5251) 2019-07-30 11:22:47 +08:00
Joey Jiang 40395acadf [gRPC] Migrate raylet client implementation to grpc (#5120) 2019-07-25 14:48:56 +08:00
Qing Wang a3d4f9f16d Fix the issue when passing multiple options in one string (#5241)
* Fix

* Fix linting

* Fix linting

* Address

* Fix test
2019-07-23 12:28:54 +08:00
Kai Yang 806524384b [Java worker] Refactor object store and worker context on top of core worker (#5079) 2019-07-16 20:58:02 +08:00
Edward Oakes e5be5fd46d Remove dependencies from TaskExecutionSpecification (#5166) 2019-07-15 18:15:21 -07:00
Hao Chen 0ec3a16bbd Fix Java MultithreadingTest (#5182) 2019-07-12 19:00:13 +08:00
Kai Yang d8b50a5018 Fix GcsClient resource map (#5171) 2019-07-11 16:05:12 +08:00
Qing Wang f2293243cc [ID Refactor] Shorten the length of JobID to 4 bytes (#5110)
* WIP

* Fix

* Add jobid test

* Fix

* Add python part

* Fix

* Fix tes

* Remove TODOs

* Fix C++ tests

* Lint

* Fix

* Fix exporting functions in multiple ray.init

* Fix java test

* Fix lint

* Fix linting

* Address comments.

* FIx

* Address and fix linting

* Refine and fix

* Fix

* address

* Address comments.

* Fix linting

* Fix

* Address

* Address comments.

* Address

* Address

* Fix

* Fix

* Fix

* Fix lint

* Fix

* Fix linting

* Address comments.

* Fix linting

* Address comments.

* Fix linting

* address comments.

* Fix
2019-07-11 14:25:16 +08:00
Hao Chen 88365d4112 Fix Java MultithreadingTest (#5170) 2019-07-11 13:40:40 +08:00
Kai Yang 43b6513d19 [GCS] Move node resource info from client table to resource table (#5050) 2019-07-11 13:17:19 +08:00