wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-07-14 11:17:54 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,843 Commits 36 Branches 42 Tags
fe65f9fbbc1e2a59dd2d11cf3fe17e3384ab746e
Commit Graph
1843 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Hao ChenandRobert Nishihara fe65f9fbbc improve java api doc (#2508) 2018-07-29 20:41:11 -07:00
Robert NishiharaandPhilipp Moritz 3f3514c2b3 Deprecate PYTHON_MODE more gracefully. (#2487) 2018-07-29 16:25:46 -07:00
Steve SeveranceandEric Liang f1b4ea69a3 Prevent hasher from running out of memory on large files (#2451) 
* Prevent hasher from running out of memory on large files

* dump out keys

* only print if failed

* remove debugging

* Fix lint error. Reverse adding newline.
 2018-07-28 23:29:09 -07:00
IonandPhilipp Moritz 80db69d245 State transition diagram documentation. (#2502) 
* Added description of transition diagram and a few name changes for imporved clarity.

* rename some methods and update task_states.rst
 2018-07-28 22:28:45 -07:00
Hao ChenandRobert Nishihara 0ea7a6abf0 add java tutorial (#2491) 2018-07-28 17:09:30 -07:00
Eric LiangandRobert Nishihara 90a3ea9443 [xray] Fix heartbeat subscription for autoscaler (#2498) 2018-07-28 13:34:55 -07:00
Peter SchafhalterandPhilipp Moritz e10377567c Add benchmark for ray.get (#2499) 2018-07-28 09:09:21 -07:00
Philipp MoritzandRobert Nishihara ecc100cb3b Upgrade arrow to include pytorch fix (#2496) 2018-07-28 01:28:44 -07:00
Peter SchafhalterandPhilipp Moritz ccb9a27393 Add benchmarks for ray.put (#2489) 2018-07-27 17:49:21 -07:00
Peter SchafhalterandPhilipp Moritz 302510ada0 [asv] Add actor benchmarks (#2469) 
* Add actor benchmarks

* Fix bug

* Address comments and refactor

* Update benchmark_actor.py
 2018-07-27 17:40:02 -07:00
Robert NishiharaandStephanie Wang 2be1ccbd8f Raise application-level exceptions for some failure scenarios. (#2429) 
* Raise application level exception for actor methods that can't be executed and failed tasks.

* Retry task forwarding for actor tasks.

* Small cleanups

* Move constant to ray_config.

* Create ForwardTaskOrResubmit method.

* Minor

* Clean up queued tasks for dead actors.

* Some cleanups.

* Linting

* Notify task_dependency_manager_ about failed tasks.

* Manage timer lifetime better.

* Use smart pointers to deallocate the timer.

* Fix

* add comment
 2018-07-27 19:53:30 -04:00
Eric LiangandGitHub 24649726dc [rllib] Use batch.count in async samples optimizer (#2488) 
Using the actual batch size reduces the risk of mis-accounting. Here, we under-counted samples since in truncate_episodes mode we were doubling the batch size by accident in policy_evaluator.
 2018-07-27 16:44:21 -07:00
Peter SchafhalterandPhilipp Moritz 1e6b130b90 [asv] Add benchmarks for tasks (#2471) 
* Add benchmarks for tasks

* Fixes

* Add timeout
 2018-07-27 13:59:55 -07:00
Stephanie WangandRobert Nishihara 6675361684 [xray] Track ray.get calls as task dependencies (#2362) 2018-07-27 11:59:17 -07:00
Hao ChenandRobert Nishihara 5b015f9a79 Remove the check of java primitive types (#2495) 2018-07-27 11:44:19 -07:00
ShuoandRobert Nishihara 29451cca82 Add test: running a driver for twice. (#2464) 2018-07-27 00:57:52 -07:00
Zhijun FuandStephanie Wang 9ad6a973a0 [xray] lineage optimization: avoid unnecessary lineage entry allocation & free (#2463) 
* merge from ray

* Revert "merge from ray"

This reverts commit 32b181ebbb1fa184026631e1a7368112c4c3118d.

* [xray] avoid unnecessary lineage entry allocation & free

* address comments

* address review comments

* address comments
 2018-07-26 10:44:38 -04:00
Yuhong GuoandRobert Nishihara 46351957bb Fix MAC_WHEELS=1 (#2477) 2018-07-25 14:57:28 -07:00
Yuhong GuoandPhilipp Moritz b35ce5dbf1 Update Arrow Package with breaking changes (#2440) 
* Merge the breaking change of Arrow Package.

* Fix typo

* Fix lint.

* put forward declarations into header

* fix

* add protocol.h

* fix linting
 2018-07-25 14:28:33 -07:00
Richard LiawandGitHub 7edc677304 [rllib] Extra Changes for Usability (#2363) 2018-07-24 20:51:22 -07:00
Sergey KolesnikovandEric Liang 05490b8cb9 [rllib] dqn/ddpg policy customization (#2445) 
* dqn policy update - more customization

* docs for custom DQN graph

* Update rllib-training.rst

* Update rllib-models.rst

* Update rllib.rst

* Update rllib-training.rst

* Update rllib-concepts.rst

* yapf codestyle
 2018-07-22 14:47:14 -07:00
Eric LiangandGitHub 68660453e4 [rllib] Better support and add two-trainer example for multiagent (#2443) 
This adds a simple DQN+PPO example for multi-agent. We don't do anything fancy here, just syncing weights between two separate trainers. This potentially is wasting some compute, but is very simple to set up.

It might be nice to share experience collection between the top-level trainers in the future.
 2018-07-22 05:09:25 -07:00
ShuoandRobert Nishihara 99d0d96aef Use different serialization context for each driver. (#2406) 2018-07-20 23:42:49 -07:00
Hao ChenandRobert Nishihara 05f485e274 Allow Ray API to be used from multiple threads (#2422) 2018-07-20 15:39:01 -07:00
Robert NishiharaandPhilipp Moritz 4b6157ed09 Remove link to install Linux Python 3.3 wheel. (#2434) 2018-07-20 15:15:43 -07:00
nam-cernandPhilipp Moritz c0b4c3b6cf Use absolute path to get to thirdparty dir (#2442) 
* Use absolute path to get to thirdparty dir

In case this script is executed from a different directory than the Ray's directory, the `pushd` will fail. This commit uses absolute path to `thirdparty` directory.

* Update setup_thirdparty.sh
 2018-07-20 15:12:25 -07:00
Eric LiangandGitHub 807f309b3a [test] Fix broken rllib test (#2446) 
This fixes the broken build.
 2018-07-20 13:47:41 -07:00
Philipp MoritzandRobert Nishihara e821f852ef [xray] Silence some object manager logging (#2437) 2018-07-20 13:10:03 -07:00
Peter SchafhalterandPhilipp Moritz 2a3b02649a Add queue test to xray tests (#2433) 2018-07-19 17:18:13 -07:00
Peter SchafhalterandPhilipp Moritz 400a3e5705 Add queue size and __len__ methods (#2432) 2018-07-19 17:04:42 -07:00
Peter SchafhalterandPhilipp Moritz 4225ac5081 Add benchmark using queue (#2431) 2018-07-19 16:43:22 -07:00
Eric LiangandGitHub 8e75d150f7 [rllib] Apex crash when compress_observations: False (#2426) 
We shouldn't try to decompress uncompressed data.

Also, fix resource requests for ddpg + GPU.
 2018-07-19 15:58:09 -07:00
Eric LiangandGitHub d01dc9e22d [rllib] format with yapf (#2427) 
* initial yapf

* manual fix yapf bugs
 2018-07-19 15:30:36 -07:00
Robert NishiharaandPeter Schafhalter 24eb140e07 Remove redundant reconstruct call. (#2421) 2018-07-19 11:22:02 -07:00
Robert NishiharaandPhilipp Moritz eed39163f9 Add callback to node manager for client removed event. (#2417) 
* Add callback to node manager for client removed event.

* Fix linting.
 2018-07-18 16:59:04 -07:00
Robert NishiharaandPhilipp Moritz 991d0911d1 Move profile data flushing to background thread on workers. (#2415) 
* Move profile data flushing to background thread on workers.

* Remove outdated comment.
 2018-07-18 12:34:53 -07:00
Philipp MoritzandRobert Nishihara 4c82ac72df Upgrade arrow to include the plasma TensorFlow op (#2412) 2018-07-18 12:33:02 -07:00
Wang QingandRobert Nishihara 344e3d2c05 Fix bug: Init RayLog before using it. (#2408) 2018-07-18 00:44:37 -07:00
Eric LiangandGitHub f31a6ca965 [rllib] Count actual sample batch size instead of configured batch size in A3C. (#2399) 
This fixes a metrics accounting bug where the sample count is not reported correctly.
 2018-07-18 08:59:52 +02:00
Richard LiawandGitHub 8e8c733696 [tune] Fix Categorical Space + Add Keras Example (#2401) 
Previously did not properly resolve categorical variables for HyperOpt.
 2018-07-17 23:52:52 +02:00
Yuhong GuoandPhilipp Moritz e3badb9b09 Fix that parquet and arrow will build every time. (#2405) 
* Fix the bug that parquet and arrow will build every time.

* Update build_arrow.sh

* Update build_arrow.sh
 2018-07-16 22:56:14 -07:00
Eric LiangandGitHub 0cecf6b79c [rllib] Cleanup RNN support and make it work with multi-GPU optimizer (#2394) 
Cleanup: TFPolicyGraph now automatically adds loss input entries for state_in_*, so that graph sub-classes don't need to worry about it.

Multi-GPU support:

Allow setting up model tower replicas with existing state input tensors

Truncate the per-device minibatch slices so that they are always a multiple of max_seq_len.
 2018-07-17 06:55:46 +02:00
Robert NishiharaandPhilipp Moritz 1b645fcc8b Add parameter server blog post. (#2398) 
* Saving work on parameter server blog post.

* Updates

* Updates to blog post.

* Add notes about tasks and actors.

* Updates

* Add RLlib paper link

* Update intro

* Address comments.

* More fixes.

* Clarify ray.get

* Change date

* Add @ray.remote clarification.

* Update site deployment instructions.

* Minor wording
 2018-07-16 21:51:39 -07:00
Peter SchafhalterandRobert Nishihara f5c46c7765 Add queue data structures (#2261) 2018-07-16 16:26:20 -07:00
Yuhong GuoandPhilipp Moritz 404bfc5da2 Add const to to_plasma_id function to make it usable by const ObjectID (#2404) 
* Add const to to_plasma_id to make it usable by const ObjectID

* Separate the building script to another PR.
 2018-07-16 11:05:51 -07:00
Yuhong GuoandPhilipp Moritz ded260b1b7 Add const to to_plasma_id function to make it usable by const ObjectID (#2404) 
* Add const to to_plasma_id to make it usable by const ObjectID

* Separate the building script to another PR.
 2018-07-16 11:05:37 -07:00
Yuhong GuoandPhilipp Moritz 206254bcf3 Add const to to_plasma_id function to make it usable by const ObjectID (#2404) 
* Add const to to_plasma_id to make it usable by const ObjectID

* Separate the building script to another PR.
 2018-07-16 11:05:29 -07:00
Hao ChenandRobert Nishihara 8a3e180156 Move profiling code to a new file and fix thread safety (#2397) 2018-07-15 18:09:52 -07:00
Yuhong GuoandPhilipp Moritz bbea73155a Fix parquet missing error and improve arrow commit id changing (#2319) 
* Fix parquet missing error and improve arrow commit id changing

* Remove build cache for arrow.

* Update build_parquet.sh

* Update build_ui.sh

* Update build_arrow.sh
 2018-07-14 16:08:13 -07:00
Eric LiangandRichard Liaw 7865dbab84 [tune] Raise error if incorrect key used in config (#2400) 2018-07-15 00:25:19 +02:00