wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-07-24 13:20:22 +08:00
Code Issues Packages Projects Releases Wiki Activity
377 Commits 36 Branches 42 Tags
e06311d4151d8e5050f922cc0c8e8b5fb0e07435
Commit Graph
56 Commits
Author SHA1 Message Date
Robert NishiharaandPhilipp Moritz e06311d415 Automatically add relevant directories to Python paths of workers (#380) 
* Make ray.init set python paths of workers.

* Decouple starting cluster from copying user source code

* also add current directory to path

* Add comments about deallocation.

* Add test for new code path.
 2016-08-16 14:53:55 -07:00
Wapaul1andPhilipp Moritz 7246013008 Implement select to enable waiting for a specific number of remote objects to be ready. (#369) 2016-08-15 16:51:59 -07:00
Robert NishiharaandPhilipp Moritz b29fc0c481 Use random string for worker c++ logfile. (#378) 2016-08-15 15:55:34 -07:00
Robert NishiharaandPhilipp Moritz e94ed2fc97 Throw fatal error when attempting to send message on full queue. (#377) 2016-08-15 11:51:23 -07:00
Robert NishiharaandPhilipp Moritz 87bb7a8f67 [WIP] Large changes to make the tests pass. (#376) 
* Revert "Make tests more informative (#372)"

This reverts commit fd353250c8.

* fix bugs, in particular deactivate worker service on driver and remove condition variables

* changes to minimize the changes in this PR

* switch from faulty mutex synchronization to using atomics

* Increase the default size of the message queues, to accommodate exporting large numbers of remote functions. This is a temporary fix, but not a long term solution.

* Reorganize the scheduler export code to queue up exports. This does not solve the underlying problem yet, but sets up a solution.

* Start a separate thread on driver to print error messages by constantly querying the scheduler. This is a temporary solution because the solution based on starting a worker service for the driver which the scheduler can push error messages to is buggy.

* Fix segfault in taskcapsule destructor.

* Move tests for catching errors into a separate test file.

* Revert "roll back grpc (#368)"

This reverts commit c01ef95d04.
 2016-08-15 11:02:54 -07:00
Philipp MoritzandRobert Nishihara b0ecff69ad Check for GRPC status codes (#366) 
* check for GRPC status codes

* more checkss
 2016-08-11 11:10:00 -07:00
Wapaul1andPhilipp Moritz 362ffa1f3c Changing hard coded ports for objstore and workers to choose unused ports (#365) 
* let grpc choose unused worker and object store ports

* Add objstore addresses to scheduler info to bring back test
 2016-08-10 19:08:38 -07:00
Robert NishiharaandPhilipp Moritz fbc49410ec properly shutdown worker service thread (#367) 2016-08-10 16:53:24 -07:00
Robert NishiharaandPhilipp Moritz a1e4268d37 Catch errors in importing reusable variables and remote functions (#354) 
* catch errors in importing reusable variables and remote functions

* updates
 2016-08-07 13:53:33 -07:00
Robert NishiharaandPhilipp Moritz ac363bf451 Let worker get worker address and object store address from scheduler (#350) 2016-08-04 17:47:08 -07:00
Robert NishiharaandPhilipp Moritz 98a508d6ca Terminology change Object Reference -> Object ID (#330) 2016-07-31 19:58:03 -07:00
mehrdadnandRobert Nishihara 36fff9b555 Retry registering workers better (#300) 2016-07-29 00:23:31 -07:00
Robert NishiharaandPhilipp Moritz 8e9f98c5ff Properly import remote functions and reusable variables on workers that register late (#290) 2016-07-25 16:17:17 -07:00
Philipp MoritzandRobert Nishihara 5591aa4665 cleanup serialization code (#291) 2016-07-25 15:47:10 -07:00
Philipp MoritzandRobert Nishihara 4a0f35b042 new arrow serialization code (serialize python objects recursively) (#284) 2016-07-25 13:41:47 -07:00
Robert Nishihara 03f1830cd0 implement key value store for sharing reusable variables 2016-07-22 18:43:33 -07:00
Wapaul1andPhilipp Moritz b68512ec7d Added handles for python to access buffers in objstore (#282) 2016-07-20 17:36:15 -07:00
mehrdadnandRobert Nishihara f2c43bec87 Function serialization (#261) 2016-07-17 22:05:07 -07:00
mehrdadnandPhilipp Moritz 5412d3c773 Call vec.data() instead of &vec[0] (#219) 2016-07-06 16:46:06 -07:00
mehrdadnandPhilipp Moritz 1de824628d Replace boost::interprocess with simple IPC (#189) 2016-06-30 16:31:48 -07:00
mehrdadnandPhilipp Moritz 5328890001 Kill workers (#148) 2016-06-29 22:40:04 -07:00
Robert NishiharaandPhilipp Moritz 72f22a238a let scheduler distinguish drivers from workers (#185) 2016-06-29 21:43:06 -07:00
Philipp Moritz 891dccb0ff Set the workerid in submit_task and get_objref 2016-06-28 21:41:31 -07:00
mehrdadnandPhilipp Moritz 27bd859925 Remove objstore_stub_ (#176) 2016-06-27 13:16:35 -07:00
Philipp Moritz 8a0b8e8789 Unmap segments properly to avoid memory leaks 2016-06-24 16:56:26 -07:00
Robert Nishihara 188569be37 push/pull -> put/get 2016-06-23 12:58:48 -07:00
Philipp Moritz 5281bd414f preparation to deallocate objects properly 2016-06-21 13:46:38 -07:00
mehrdadnandPhilipp Moritz 0e5feecd65 Get failed worker information (#114) 2016-06-17 15:01:48 -07:00
Robert NishiharaandPhilipp Moritz 102e3682d9 make sure the scheduler sends tasks to the worker only after the worker is ready (#116) 2016-06-16 16:04:52 -07:00
Mehrdad N e8ed13e960 Unregistered function handling 2016-06-13 23:30:10 +00:00
Philipp Moritz 7585d5eab3 Downgrading reference count logging to DEBUG 2016-06-13 15:04:10 -07:00
Wapaul1 4e885d4896 Added RAY_CHECK with line num and file name for logs 2016-06-13 00:07:22 +00:00
Robert NishiharaandPhilipp Moritz 41539141af catch exceptions on workers and pass them to the scheduler (#93) 2016-06-11 15:44:56 -07:00
Robert NishiharaandPhilipp Moritz 4cc024ae36 renaming project, halo -> ray (#95) 2016-06-10 14:12:15 -07:00
Robert NishiharaandPhilipp Moritz 67086f663e change filenames and directory structure to use halo (#81) 2016-06-03 18:32:57 -07:00
Robert NishiharaandPhilipp Moritz 23d202bb73 renaming Call -> Task (#70) 2016-05-26 16:33:30 -07:00
Robert NishiharaandPhilipp Moritz c4b783fbc8 microbenchmarks and timing (#67) 2016-05-22 10:05:14 -07:00
Philipp Moritz d3f62c4288 make it possible to start multiple nodes in a cluster; add some more logging 2016-04-28 15:31:31 -07:00
Philipp Moritz 8d0c3be28f implement serialization using numbuf 2016-04-26 15:14:04 -07:00
Philipp Moritz e02d246974 reduce the amount of logging, separating out logging for reference counting 2016-04-22 12:54:36 -07:00
Robert NishiharaandPhilipp Moritz 87bc8801ab changing segment names to include objstoreid (#50) 2016-04-22 12:07:02 -07:00
Robert NishiharaandPhilipp Moritz 1548a1a523 implement reference counting and much more (#43) 2016-04-18 13:05:36 -07:00
Robert NishiharaandPhilipp Moritz a28920bb24 Alias (#38) 
* implement objref aliasing

* updates
 2016-04-08 12:58:08 -07:00
Robert Nishihara 6a5eeec86b fix naming Pull -> Request 2016-04-06 22:06:53 -07:00
Philipp Moritz f5147d8e59 arrow integration for ndarrays 2016-04-05 00:57:14 -07:00
Philipp Moritz c1459f3603 clean up queues 2016-03-17 22:56:55 -07:00
Philipp Moritz dd88bae487 implement event based object store 2016-03-17 22:32:31 -07:00
Robert Nishihara b773003b70 make ObjStore.GetObj wait until the object is present in the object store, other miscellaneous fixes 2016-03-15 00:13:12 -07:00
Philipp Moritz c249efa7d1 Improved OrchPy lib error handling, fixed tests, added tests 2016-03-10 20:10:49 -08:00
Philipp Moritz 9440e0f08a backend: make objectstores more robust, add logging, add reporting back of workers 2016-03-10 16:55:56 -08:00