Commit Graph

34 Commits

Author SHA1 Message Date
Philipp Moritz 869ee8e25d Integrate plasma store list facility (#2752) 2018-09-01 16:53:51 -07:00
Philipp Moritz b4c47a5861 Upgrade arrow to include more detailed flushing message (#2706) 2018-08-24 11:44:04 -07:00
Yuhong Guo 344a83f327 Fix build failure of Arrow and Parquet when the folder is empty. (#2720) 2018-08-23 09:44:26 -07:00
Hao Chen dd924a388b silence progress log from 'git clone' and 'pip install' (#2667) 2018-08-15 22:54:35 -07:00
Hao Chen 3c75e71afc reduce noisy log messages from wget (#2656) 2018-08-15 09:10:28 -07:00
Philipp Moritz f13e3e22f2 Upgrade arrow to include tensorflow op fix (#2607) 2018-08-14 21:47:01 -07:00
Yuhong Guo 4bd98eed45 Support building Java and Python version at the same time. (#2640)
* Support building Java and Python version at the same time.

* Remove duplicated definition.

* Refine the building process of local_scheduler

* Refine

* Add comment for languages

* Modify instruction and add python,jave building to CI.

* change according to comment
2018-08-14 11:33:51 -07:00
Philipp Moritz 5e59cc6a20 Update arrow to include plasma memory footprint reduction (#2545) 2018-08-02 14:37:37 -07:00
Philipp Moritz d9a019b8e5 Upgrade arrow to include pytorch fix (#2522)
This fixes https://github.com/ray-project/ray/issues/2520
2018-07-30 20:20:18 -07:00
Philipp Moritz ecc100cb3b Upgrade arrow to include pytorch fix (#2496) 2018-07-28 01:28:44 -07:00
Yuhong Guo b35ce5dbf1 Update Arrow Package with breaking changes (#2440)
* Merge the breaking change of Arrow Package.

* Fix typo

* Fix lint.

* put forward declarations into header

* fix

* add protocol.h

* fix linting
2018-07-25 14:28:33 -07:00
Philipp Moritz 4c82ac72df Upgrade arrow to include the plasma TensorFlow op (#2412) 2018-07-18 12:33:02 -07:00
Yuhong Guo e3badb9b09 Fix that parquet and arrow will build every time. (#2405)
* Fix the bug that parquet and arrow will build every time.

* Update build_arrow.sh

* Update build_arrow.sh
2018-07-16 22:56:14 -07:00
Yuhong Guo bbea73155a Fix parquet missing error and improve arrow commit id changing (#2319)
* Fix parquet missing error and improve arrow commit id changing

* Remove build cache for arrow.

* Update build_parquet.sh

* Update build_ui.sh

* Update build_arrow.sh
2018-07-14 16:08:13 -07:00
Philipp Moritz 4dadc60968 Update arrow to include uninitialized memory fixes (#2371) 2018-07-11 07:52:02 -05:00
Philipp Moritz e45bec1e51 Fix import error when tensorflow-gpu and tensorflow are both installed (#2364) 2018-07-06 22:29:54 -07:00
Philipp Moritz fbde8cad74 Update apache arrow to include TensorFlow fix (#2345) 2018-07-06 13:18:56 -07:00
Philipp Moritz 4f1d14e176 improve error message when flex or bison is not installed (#2355) 2018-07-05 23:46:55 -07:00
Philipp Moritz c0c7146dec fix Travis linux build (#2308) 2018-06-26 20:19:07 -10:00
Yuhong Guo aa42331844 Fix build failure while using make -j1. Issue 2257 (#2279)
* Fix build failure while using make -j1

* Fix java test failure
2018-06-21 15:18:00 -07:00
Zongheng Yang 8190ff1fd0 Experimental: enable automatic GCS flushing with configurable policy. (#2266)
* build_credis.sh: use an up-to-date credis commit.

* build_credis.sh: leveldb is updated, so update build cmds for it

* WIP: make monitor.py issue flush; switch gcs client to use credis

* Experimental: enable automatic GCS flushing with configurable policy.

* Fix linux compilation error

* Fix leveldb build

* Use optimized build for credis

* Address comments

* Attempt to fix tests
2018-06-20 14:40:57 -07:00
songqing 78a48fa1e0 Fix build error when building Ray for Java later than Python (#2241) 2018-06-12 21:11:30 -07:00
songqing 4dd4698564 unify build dir for Python and Java (#2171)
* unify build dir for Python and Java

* enable executables auto installed when just running 'make'

* fix plasma_store copy error

* fix cmake error about copying executables

* lint fix

* recover python/setup.py

* enable to copy optional file automatically

* a small fix of path

* lint fix

* lint fix

* lint fix

* Add comment.
2018-06-01 16:28:27 -07:00
Yujie Liu a8d3c057c1 [JavaWorker] Enable java worker support (#2094)
* Enable java worker support
--------------------------
This commit includes a tailored version of the Java worker implementation from Ant Financial.
The changes for build system, python module, src module and arrow are in other commits, this commit consists of the following modules:
 - java/api: Ray API definition
 - java/common: utilities
 - java/hook: binary rewrite of the Java byte-code for remote execution
 - java/runtime-common: common implementation of the runtime in worker
 - java/runtime-dev: a pure-java mock implementation of the runtime for fast development
 - java/runtime-native: a native implementation of the runtime
 - java/test: various tests

Contributors for this work:
 Guyang Song, Peng Cao, Senlin Zhu,Xiaoying Chu, Yiming Yu, Yujie Liu, Zhenyu Guo

* change the format of java help document from markdown to RST

* update the vesion of Arrow for java worker

* adapt the new version of plasma java client from arrow which use byte[] instead of custom type

* add java worker test to ci

* add the example module for better usage guide
2018-05-26 14:38:50 -07:00
Zongheng Yang fa97acbc89 Integrate credis with Ray & route task table entries into credis. (#1841) 2018-05-24 23:35:25 -07:00
Philipp Moritz 225608ec66 Update arrow to latest master (#2100) 2018-05-24 00:26:13 -07:00
Philipp Moritz af88fdefcf Incorporate C++ Buffer management and Seal global threadpool fix from arrow (#1950) 2018-04-25 22:53:44 -07:00
Robert Nishihara 24c944e499 Update arrow to efficiently serialize more types of numpy arrays. (#1889) 2018-04-13 09:41:51 -07:00
Robert Nishihara d0fffec2d0 Update arrow and parquet-cpp. (#1875)
* Update arrow.

* Fix bug.

* Cherry-pick commit for fixing parquet segfault.

* Update arrow and revert auto-releasing buffer commit.

* Remove parquet cherry-pick.
2018-04-12 16:17:12 -07:00
alonamid 202f9683ea check if arrow build dir exists (#1863) 2018-04-10 14:52:51 -07:00
Robert Nishihara d8adfbd440 Fail build if pyarrow hasn't successfully been built. (#1721) 2018-03-16 12:02:36 -07:00
Philipp Moritz 5ef0892236 Compile boost from source to fix macOS wheels (#1688) 2018-03-08 23:22:23 -08:00
Philipp Moritz a9acfab3a6 Start chain replicated GCS with Ray (#1538) 2018-03-07 10:18:58 -08:00
Zhenyu Guo f1e5789c26 restructure how to organize 3rd party libs (#1630)
* restructure how to organize 3rd party libs

* Minor whitespace changes.

* Fix compilation on Linux.

* Pass around Python executable so that the correct version of Python is used.
2018-03-01 14:29:56 -08:00