Commit Graph

72 Commits

Author SHA1 Message Date
mehrdadn 0a54407961 [CI] Factor out more Travis code and update GitHub Actions (#8085) 2020-04-21 09:53:08 -07:00
mehrdadn ba00c29b67 Factor out Travis 'install' sections for use with GitHub Actions (#7988) 2020-04-15 08:10:22 -07:00
mehrdadn 65054a2c7c Python 3.8 compatibility (#7754) 2020-04-01 10:03:23 -07:00
Simon Mo b804d40c04 Stop vendoring pyarrow (#7233) 2020-02-19 19:01:26 -08:00
Simon Mo 7bef7031c2 Revert "Revert "Revert "Removing Pyarrow dependency (#7146)" (#7209) (#7214)" (#7232) 2020-02-19 13:35:29 -08:00
Simon Mo e8941b1b79 Revert "Revert "Removing Pyarrow dependency (#7146)" (#7209) (#7214) 2020-02-19 10:08:52 -08:00
Eric Liang 0aa9373d62 Revert "Removing Pyarrow dependency (#7146)" (#7209)
This reverts commit 2116fd3bca.
2020-02-18 14:12:06 -08:00
ijrsvt 2116fd3bca Removing Pyarrow dependency (#7146) 2020-02-17 18:00:13 -08:00
Siyuan (Ryans) Zhuang 6745459f96 Apply cpython patch bpo-39492 for the reference counting issue in pickle5 (#7177)
* apply cpython patch bpo-39492 for the reference count issue
2020-02-15 21:16:13 -08:00
Eric Liang 58c94f6381 [core] Delete() should never remote objects from in-memory store (#7117) 2020-02-10 22:40:09 -08:00
Dean Wampler c729378bd5 Check that the user's python version is supported (#7003) 2020-02-10 10:32:23 -08:00
ijrsvt 0826f95e1c Including psutil & setproctitle (#7031) 2020-02-05 14:16:58 -08:00
Siyuan (Ryans) Zhuang 42cbf801e1 workaround for python3.5 fast numpy serialization (#6675) 2020-02-03 13:08:18 -08:00
chaokunyang 4097d076d4 Package ray java jars into wheels (#6600) 2020-01-10 11:41:00 +08:00
Philipp Moritz decaa65cd6 Use pickle by default for serialization (#5978) 2019-11-10 18:12:18 -08:00
Philipp Moritz ccbcc4bafa Use GRCP and Bazel 1.0 (#6002) 2019-11-08 15:58:28 -08:00
Philipp Moritz 1c5446851a Use Plasma with LRU refreshing integrated (#6050) 2019-11-03 16:19:05 -08:00
mehrdadn e706cb63cc Fix missing double quotes for spaces in paths (#6026) 2019-10-26 20:46:55 -07:00
Philipp Moritz 010270b3dc Cleanup left over shell scripts in build process (#6017) 2019-10-26 15:46:46 -07:00
Eric Liang 4bf7de084d Speed up TaskSpecification copy (#5709) 2019-09-15 19:57:34 -07:00
Eric Liang e2e30ca507 Ray, Tune, and RLlib support for memory, object_store_memory options (#5226) 2019-08-21 23:01:10 -07:00
Simon Mo 1f8ae17f60 Silence some installation process for build from source (#5396) 2019-08-07 16:54:48 -07:00
Richard Liaw 9c00616cdc Retry and exception for hang on memory store full (#5143) 2019-07-27 01:20:13 -07:00
Philipp Moritz 322b5166ad Update arrow to include user defined status for plasma (#5156) 2019-07-12 22:51:14 -07:00
Philipp Moritz 4e82313891 Update to latest arrow (#5011) 2019-06-30 20:36:36 -07:00
Philipp Moritz 64eb7b322c Upgrade arrow to latest master (#4858) 2019-05-28 16:04:16 -07:00
Robert Nishihara af463e8bb1 Find bazel even if it isn't in the PATH. (#4729) 2019-05-01 21:29:48 -07:00
Hao Chen be2cbdf130 [Bazel] Use rules_jvm_external to manage java dependencies (#4615) 2019-04-18 16:53:25 +08:00
Philipp Moritz b0f6ddf6d1 Remove CMake files (#4493) 2019-04-02 22:17:33 -07:00
Philipp Moritz 1bcb0b94cc Synchronize arrow version and put changes upstream (#4385) 2019-03-27 22:37:07 -07:00
Hao Chen 7a38f9be1c Avoid redundant bazel build (#4458) 2019-03-23 10:44:11 +08:00
Ruifang Chen 59d74d5e92 [Java] Build Java code with Bazel (#4284) 2019-03-22 14:30:05 +08:00
Philipp Moritz 490d896f41 Make sure the right Python interpreter is used (#4334) 2019-03-12 12:21:55 -07:00
Philipp Moritz dec7c3f8f5 [build] Add debug info to Bazel (#4278) 2019-03-07 15:21:13 -08:00
Philipp Moritz 180414710e Make sure Bazel generated files get overwritten (#4205) 2019-03-02 13:38:37 -08:00
Philipp Moritz 4dc683d39e Use latest arrow wheels (#4182) 2019-02-28 12:17:32 -08:00
Philipp Moritz ba52caff37 Make Bazel the default build system (#3898) 2019-02-23 11:58:59 -08:00
Wang Qing 3a7fb182cc Change the num of parallel jobs when building 2019-02-13 00:33:05 -08:00
Philipp Moritz ce6e01b988 enable incremental builds (#3292) 2018-11-12 21:49:09 -05:00
Eric Liang 9b2794101d [minor] Change chunk already exists to DEBUG, add flags for rllib multi node testing (#3228) 2018-11-08 00:04:20 -08:00
Hanwei Jin 87639b9e26 move make clean before cmake command, avoid always running mvn install plasma java lib (#3047) 2018-10-12 09:03:30 -07:00
Hanwei Jin fbf214e408 update ray cmake build process (#2853)
* use cmake to build ray project, no need to appply build.sh before cmake, fix some abuse of cmake, improve the build performance

* support boost external project, avoid using the system or build.sh boost

* keep compatible with build.sh, remove boost and arrow build from it.

* bugfix: parquet bison version control, plasma_java lib install problem

* bugfix: cmake, do not compile plasma java client if no need

* bugfix: component failures test timeout machenism has problem for plasma manager failed case

* bugfix: arrow use lib64 in centos, travis check-git-clang-format-output.sh does not support other branches except master

* revert some fix

* set arrow python executable, fix format error in component_failures_test.py

* make clean arrow python build directory

* update cmake code style, back to support cmake minimum version 3.4
2018-09-12 11:19:33 -07:00
Philipp Moritz b4c47a5861 Upgrade arrow to include more detailed flushing message (#2706) 2018-08-24 11:44:04 -07:00
Yuhong Guo 4bd98eed45 Support building Java and Python version at the same time. (#2640)
* Support building Java and Python version at the same time.

* Remove duplicated definition.

* Refine the building process of local_scheduler

* Refine

* Add comment for languages

* Modify instruction and add python,jave building to CI.

* change according to comment
2018-08-14 11:33:51 -07:00
songqing 4dd4698564 unify build dir for Python and Java (#2171)
* unify build dir for Python and Java

* enable executables auto installed when just running 'make'

* fix plasma_store copy error

* fix cmake error about copying executables

* lint fix

* recover python/setup.py

* enable to copy optional file automatically

* a small fix of path

* lint fix

* lint fix

* lint fix

* Add comment.
2018-06-01 16:28:27 -07:00
Yujie Liu a8d3c057c1 [JavaWorker] Enable java worker support (#2094)
* Enable java worker support
--------------------------
This commit includes a tailored version of the Java worker implementation from Ant Financial.
The changes for build system, python module, src module and arrow are in other commits, this commit consists of the following modules:
 - java/api: Ray API definition
 - java/common: utilities
 - java/hook: binary rewrite of the Java byte-code for remote execution
 - java/runtime-common: common implementation of the runtime in worker
 - java/runtime-dev: a pure-java mock implementation of the runtime for fast development
 - java/runtime-native: a native implementation of the runtime
 - java/test: various tests

Contributors for this work:
 Guyang Song, Peng Cao, Senlin Zhu,Xiaoying Chu, Yiming Yu, Yujie Liu, Zhenyu Guo

* change the format of java help document from markdown to RST

* update the vesion of Arrow for java worker

* adapt the new version of plasma java client from arrow which use byte[] instead of custom type

* add java worker test to ci

* add the example module for better usage guide
2018-05-26 14:38:50 -07:00
Yujie Liu 5918776dd4 [JavaWorker] Changes to the build system for support java worker (#2092)
* Changes to the build system for support java worker
--------------------------
This commit includes changes to the build system, which is part of the java worker support of Ray.
It consists of the following changes:
 - the changes of CMakeLists.txt files
 - the changes of the python setup.py and init files for the adaptation of the changed build system
 - move the location of local_scheduler_extension.cc for the adaptation of the changed build system which maybe better support multi-language worker

* minor whitespace

* Linting
2018-05-18 19:09:23 -07:00
Zhenyu Guo f1e5789c26 restructure how to organize 3rd party libs (#1630)
* restructure how to organize 3rd party libs

* Minor whitespace changes.

* Fix compilation on Linux.

* Pass around Python executable so that the correct version of Python is used.
2018-03-01 14:29:56 -08:00
Philipp Moritz a3f8fa426b Start integrating new GCS APIs (#1379)
* Start integrating new GCS calls

* fixes

* tests

* cleanup

* cleanup and valgrind fix

* update tests

* fix valgrind

* fix more valgrind

* fixes

* add separate tests for GCS

* fix linting

* update tests

* cleanup

* fix python linting

* more fixes

* fix linting

* add plasma manager callback

* add some documentation

* fix linting

* fix linting

* fixes

* update

* fix linting

* fix

* add spillback count

* fixes

* linting

* fixes

* fix linting

* fix

* fix

* fix
2018-01-31 11:01:12 -08:00
Philipp Moritz 1290072764 [rllib] Expose PPO evaluator resource requirements (#1391) 2018-01-11 11:09:01 -08:00