wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-07-25 13:30:52 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,110 Commits 36 Branches 42 Tags
6531eed2d0b17e26bab2ea60fb9de6e659610098
Commit Graph
2110 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Eric LiangandRichard Liaw 6531eed2d0 [rllib] Better error message when action space dim too high (#3119) 2018-10-26 16:55:00 -07:00
Robert NishiharaandPhilipp Moritz 658c14282c Remove legacy Ray code. (#3121) 
* Remove legacy Ray code.

* Fix cmake and simplify monitor.

* Fix linting

* Updates

* Fix

* Implement some methods.

* Remove more plasma manager references.

* Fix

* Linting

* Fix

* Fix

* Make sure class IDs are strings.

* Some path fixes

* Fix

* Path fixes and update arrow

* Fixes.

* linting

* Fixes

* Java fixes

* Some java fixes

* TaskLanguage -> Language

* Minor

* Fix python test and remove unused method signature.

* Fix java tests

* Fix jenkins tests

* Remove commented out code.
 2018-10-26 13:36:58 -07:00
Eric LiangandGitHub 055daf17a0 [autoscaler] better message if there are more than 10 key pairs 2018-10-26 12:42:11 -07:00
bibabolynnandHao Chen b4614ae69a [java] customize path of ray.conf (#3100) 
users can add custom path of ray.config by using -Dray.config=/path/to/ray.conf
 2018-10-26 13:36:34 +08:00
Philipp MoritzandRobert Nishihara d3148cc3ab [SGD] Provide better error message if model graphs have different numbers of variables (#3139) 2018-10-25 22:18:10 -07:00
Philipp MoritzandRobert Nishihara d34516f1f8 Update Gemfile Jekyll version (#3140) 2018-10-25 21:43:08 -07:00
Robert NishiharaandPhilipp Moritz 5aa29613db Fix linting errors. (#3127) 2018-10-24 16:30:00 -07:00
Eric LiangandGitHub 55d161b49f [autoscaler] Also grant roles to worker nodes 2018-10-24 13:57:36 -07:00
Hanwei JinandHao Chen 7c1fd19fd9 [Java] support python worker command in raylet (#3092) 
<!--
Thank you for your contribution!

Please review https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before opening a pull request.
-->

## What do these changes do?

support raylet, which is started by java runManager, to start python default_worker.py . 

So when doing local test of java call python task, it helps auto start python worker.

## Related issue number

<!-- Are there any issues opened that will be resolved by merging this change? -->
 2018-10-24 20:43:39 +08:00
Robert NishiharaandPhilipp Moritz 9c1826ed69 Use XRay backend by default. (#3020) 
* Use XRay backend by default.

* Remove irrelevant valgrind tests.

* Fix

* Move tests around.

* Fix

* Fix test

* Fix test.

* String/unicode fix.

* Fix test

* Fix unicode issue.

* Minor changes

* Fix bug in test_global_state.py.

* Fix test.

* Linting

* Try arrow change and other object manager changes.

* Use newer plasma client API

* Small updates

* Revert plasma client api change.

* Update

* Update arrow and allow SendObjectHeaders to fail.

* Update arrow

* Update python/ray/experimental/state.py

Co-Authored-By: robertnishihara <robertnishihara@gmail.com>

* Address comments.
 2018-10-23 12:46:39 -07:00
Robert NishiharaandEric Liang 9d2e864caf Fix Python linting error. (#3113) 2018-10-22 23:41:42 -07:00
Robert NishiharaandPhilipp Moritz 22dd7e0428 Add test for wait reconstruction. (#3110) 2018-10-22 23:16:54 -07:00
Eric LiangandGitHub 73a092e08c update (#3112) 2018-10-22 22:55:43 -07:00
Philipp MoritzandRobert Nishihara 8d8b6e5bfa Retry connections to redis for async and subscribe contexts (#3105) 
This is fixing a problem that @devin-petersohn observed on the windows subsystem for linux.

In theory, redis should be up once the async connect is happening and there should be no retries needed for the async connect. However on the windows subsystem for linux, the async connect was failing even though the synchronous one was working. Maybe windows has a different semantics here than linux.
 2018-10-22 22:31:13 -07:00
Richard LiawandEric Liang eff7cb4458 [tune] Fix SearchAlg finishing early (#3081) 
* Fix trial search alg finishing early

* Fix lint

* fix lint

* nit fix
 2018-10-22 12:17:13 -07:00
Eric LiangandGitHub 221d1663c1 [rllib] switch to python logger (#3098) 
* logg

* set rllib logger

* comment

* info

* rlib

* comment

* add format

* fix lint

* add file info

* update

* add ts

* lint

* better docs

* fix value error

* soft log level
 2018-10-21 23:43:57 -07:00
Richard LiawandRobert Nishihara 40c4148d4f Cluster Utilities for Fault Tolerance Tests (#3008) 2018-10-20 22:56:29 -07:00
Wang QingandHao Chen a4db5bbaea Fill driver id into actor notification when finishing assigned task. (#3080) 
## What do these changes do?
Fill driver id into actor notification when finishing assigned task.
Also it improves codes.
 2018-10-21 11:12:20 +08:00
Eric LiangandGitHub 59901a88a0 [rllib] Native support for Dict and Tuple spaces; fix Tuple action spaces; add prev a, r to LSTM (#3051) 2018-10-20 15:21:22 -07:00
Robert NishiharaandRichard Liaw 9a2b5333ef Add links for latest Python 3.7 wheels to documentation. (#3091) 2018-10-19 12:15:22 -07:00
bibabolynnandHao Chen 9a5c273db7 [java] fix check exception type (#3093) 
<!--
Thank you for your contribution!

Please review https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before opening a pull request.
-->

## What do these changes do?
remove TaskExecutionException, use RayException instead
<!-- Please give a short brief about these changes. -->

## Related issue number

<!-- Are there any issues opened that will be resolved by merging this change? -->
 2018-10-19 06:43:42 -07:00
Wang QingandHao Chen b410ee0d29 [Java] Support dynamically defining resources when submitting task. (#3070) 
## What do these changes do?
Before this PR, if we want to specify some resources, we must do as following codes:
```java
@RayRemote(Resources={ResourceItem("CPU", 10)})
public static void f1() {
// do sth
}

@RayRemote(Resources={ResourceItem("CPU", 10)})
class Demo {
// sth
}
```
Unfortunately, it's no way for us to create another actor or task with different resources required.

After this PR, the thing will be:
```java
ActorCreationOptions option = new ActorCreationOptions(); 
option.resources.put("CPU", 4.0);
RayActor<Echo> echo1 = Ray.createActor(Echo::new, option);
option.resources.put("Res-A", 4.0);
RayActor<Echo> echo2 = Ray.createActor(Echo::new, option);


//if we don't specify resource,  the resources will be `{"cpu":0.0}` by default.
Ray.call(Echo::echo, echo2, 100);
```


## Related issue number
N/A
 2018-10-19 06:22:32 -07:00
Eric LiangandStephanie Wang 9d23fa03c9 [xray] All messages on main asio event loop should be written asynchronously (#3023) 
* copy over ref code

* wip async writes

* compiles

* fix error handling

* add test

* amend

* fix test

* clang fmgt

* clang format

* wip

* yapf

* rename format script

* test error

* clangfmt

* add test to list

* warn

* ref test

* fix test

* comment

* add capture

* Update client_connection.cc

* wip

* fix compile
 2018-10-18 21:56:22 -07:00
Peter SchafhalterandRobert Nishihara fa469783d8 Fix bug when connecting to password-secured cluster (#3083) 2018-10-18 21:43:03 -07:00
Devin PetersohnandPhilipp Moritz 8fcdafc6ea Adding Python3.7 wheels support (#2546) 
* Adding Python3.7 wheels support

* Adding Mac wheels update

* fix

* numpy version

* choose different numpy versions depending on python version

* fix
 2018-10-18 17:58:39 -07:00
Yuhong GuoandPhilipp Moritz 653c5b114a [c++] Refine Log Code (#2816) 
* Support setting logging level from env variable

* Remove Env Variable related code

* lint
 2018-10-18 10:51:36 -07:00
Peter SchafhalterandRobert Nishihara b82fd157a7 Remove Redis protected mode (#3073) 
Follow-up to #2925 and #2952. Removes the Redis protected mode implementation from Ray which was replaced by Redis port authentication.
 2018-10-17 22:48:14 -07:00
Philipp MoritzandRobert Nishihara 2c52d9dfa0 Fix actor handle id creation when actor handle was pickled (#3074) 2018-10-17 18:00:52 -07:00
Richard LiuandEric Liang 3c0803e7e9 [rllib] use ray.wait to get next worker result in async sample optimizer (#2993) 2018-10-17 17:44:51 -07:00
Peter SchafhalterandPhilipp Moritz a41bbc10ef Add password authentication to Redis ports (#2952) 
* Implement Redis authentication

* Throw exception for legacy Ray

* Add test

* Formatting

* Fix bugs in CLI

* Fix bugs in Raylet

* Move default password to constants.h

* Use pytest.fixture

* Fix bug

* Authenticate using formatted strings

* Add missing passwords

* Add test

* Improve authentication of async contexts

* Disable Redis authentication for credis

* Update test for credis

* Fix rebase artifacts

* Fix formatting

* Add workaround for issue #3045

* Increase timeout for test

* Improve C++ readability

* Fixes for CLI

* Add security docs

* Address comments

* Address comments

* Adress comments

* Use ray.get

* Fix lint
 2018-10-16 22:48:30 -07:00
Eric LiangandGitHub a9e454f6fd [rllib] Include config dicts in the sphinx docs (#3064) 2018-10-16 15:55:11 -07:00
Wang QingandHao Chen 64e5eb305e [Java] Add jvm-parameters in Config. (#3065) 2018-10-16 15:03:18 -07:00
Praveen PalanisamyandRichard Liaw 4d8cfc0bf5 [tune] Fix (some more) misleading comments in tune/results.py (#3068) 
## What do these changes do?

Fix the misleading comments in code for:
 - `EPISODES_THIS_ITER`
 - `EPISODES_TOTAL`

Had noted it before and planned to fix it along with some other changes but seemed very relevant to stay next to #3058 so sending this now.
 2018-10-16 11:07:53 -07:00
Eric LiangandGitHub 6240ccbc6e [rllib] Add more warnings when multi-agent envs might not be set up right (#3061) 2018-10-15 13:42:56 -07:00
Eric LiangandGitHub 3c891c6ece [rllib] Parallel-data loading and multi-gpu support for IMPALA (#2766) 2018-10-15 11:02:50 -07:00
MarlonandEric Liang 4dc78b735b [tune] Fix misleading comment (#3058) 2018-10-14 22:25:39 -07:00
Eric LiangandGitHub 866c7a574c [rllib] Don't crash printing out error message (#3054) 
* fix er

* update
 2018-10-13 19:50:23 -07:00
Eric LiangandGitHub 473ee4eb3f [rllib] Add unit test and some better error messages for custom policy states (#3032) 2018-10-13 00:03:52 -07:00
Hanwei JinandPhilipp Moritz 87639b9e26 move make clean before cmake command, avoid always running mvn install plasma java lib (#3047) 2018-10-12 09:03:30 -07:00
Richard LiawandGitHub f9b58d7b02 [tune] Tweaks to Trainable and Verbosity (#2889) 2018-10-11 23:42:13 -07:00
Wang QingandHao Chen 828fe24b39 [Java] Fix loading driver resources issue. (#3046) 
## What do these changes do?
Fix the issue how we load driver resources by a specified path.
Also this addressed the comments from the related PR [3044](https://github.com/ray-project/ray/pull/3044).

## Related PRs:
 [#3044](https://github.com/ray-project/ray/pull/3044) and [#3001](https://github.com/ray-project/ray/pull/3001).
 2018-10-11 09:45:21 -07:00
Wang QingandRobert Nishihara 4a2ed47b6c [Java] Improve some Java code (#3040) 
This PR improves some java codes,  and removes some duplicated code.
 2018-10-10 17:30:23 -07:00
Hanwei JinandPhilipp Moritz 060891a9c9 [cmake] avoid to re-build pyarrow (#2963) 
* bugfix: env exists check error

* support to avoid re-build pyarrow in project

* bugfix: adapt gtest for centos lib64

* bugfix: check gtest lib exists in the directory

* bugfix: find gtest with checking all libs exists

* prefix RAY_ to thirdparty env variables to avoid conflicts with other module

* arrow use glog from ray

* change the glog and gtest install dir
 2018-10-10 14:33:15 -07:00
Wang QingandRobert Nishihara ef1f2fde95 Fix the uniqueId toString format. (#3035) 2018-10-08 13:12:14 -07:00
Wang QingandHao Chen 84bf5fc8f3 [Java] Load driver resources from local path. (#3001) 
## What do these changes do?
1. Add a configuration item `driver.resource-path`.
2. Load driver resources from the local path which is specified in the `ray.conf`.

Before this change, we should add all driver resources(like user's jar package, dependencies package and config files) into `classpath`.

After this change, we should add the driver resources into the mount path which we can configure it in `ray.conf`, and we shouldn't configure `classpath` for driver resources any more.

## Related issue number
N/A
 2018-10-08 21:05:26 +01:00
Kristian HartikainenandEric Liang 2d35a97a76 Bug/log syncer fails with parentheses (#2653) 
* Update rsync command

* Escape rsync locations

* Fix the accidental variable move

* Update rsync to use -s flag
 2018-10-06 00:34:53 -07:00
Richard LiawandRobert Nishihara ecd8f39580 [core] Improve logging message when plasma store is started. (#3029) 
Improve logging message when plasma store is started.
 2018-10-05 15:24:24 -07:00
Richard LiawandRobert Nishihara 0651d3b629 [tune/core] Use Global State API for resources (#3004) 2018-10-04 17:23:17 -07:00
Robert NishiharaandAlexey Tumanov faa31ae018 Introduce concept of resources required for placing a task. (#2837) 
* Introduce concept of resources required for placement.
* Add placement resources to task spec
* Update java worker
* Update taskinfo.java
 2018-10-04 10:35:39 -07:00
Richard LiawandRobert Nishihara 01bb073569 Suppress errors when worker or driver intentionally disconnects. (#2935) 2018-10-04 00:06:34 -07:00