Commit Graph
1598 Commits
Author SHA1 Message Date
Eric LiangandGitHub 65dcafdc3f [rllib] Refactor save() / restore() code of agents and avoid O(n_workers) save size (#2982) 2018-09-30 01:15:13 -07:00
Eric LiangandGitHub 747253e0f6 [rllib] Don't shuffle samples in PPO when using lstm 2018-09-30 01:13:56 -07:00
Eric LiangandGitHub b06c604a51 [rllib] Add some more tuned atari results to documentation (#2991)
* dqn results ++

* add scale

* hour

* fix

* small dqn table

* update

* steps

* upd

* apex

* up

* add apex results

* tip
2018-09-29 23:13:36 -07:00
Eric LiangandGitHub cf9cd5da9d [ray] Add --new flag for ray attach (#2973)
* new flag

* yapf
2018-09-29 23:04:13 -07:00
Eric LiangandRichard Liaw cb56f39070 [rllib] Entropy calculation for diag gaussian missing 0.5 term (#2968)
See: https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Entropy
2018-09-29 22:57:47 -07:00
old-bearandRichard Liaw b3f0dcf20b [tune] Add a raise_on_failed_trial flag in run_experiments (#2961)
Adds a flag to control raising TuneError if some trial fails in `run_experiments`.
2018-09-29 11:29:46 -07:00
Eric LiangandGitHub f1c55497ce [rllib] Fix edge case in n-step calculation and non-apex replay prioritization (#2929)
* fix

* lint
2018-09-28 15:22:33 -07:00
MarlonandEric Liang 5eaf429c53 Fix typo in autoscaler yaml (#2981) 2018-09-27 09:48:18 -07:00
Richard LiawandGitHub 1c9617bc1c [autoscaler] Add tmux support for attach and exec (#2907)
Adds a tmux flag that can be used to support background execution of experiments. Cannot be used together with screen. Seems to be useful feature that has shown up with different users.
2018-09-26 23:22:45 -07:00
eugenevinitskyandEric Liang 1943ae44da [rllib] Use SGD optimizer for ARS (#2916) 2018-09-26 22:32:26 -07:00
Peter SchafhalterandRobert Nishihara fcdca6de18 Fix test for available resources (#2914) 2018-09-25 23:07:23 -07:00
Robert NishiharaandEric Liang 39b4a89fde Bump version 0.5.2 to 0.5.3. (#2936) 2018-09-25 09:49:58 -07:00
Eric LiangandGitHub 75ef70afca [rllib] Auto-clip atari rewards 2018-09-24 12:55:11 -07:00
Eric LiangandGitHub 8331d1ebe0 [rllib] Add vf clipping param to fix pendulum example (#2921)
* add vf clip

* fix test

* Update ppo.py
2018-09-23 13:11:17 -07:00
Eric LiangandPhilipp Moritz 3267676994 [Experimental] Add experimental distributed SGD API (#2858)
* check in sgd api

* idx

* foreach_worker foreach_model

* add feed_dict

* update

* yapf

* typo

* lint

* plasma op change

* fix plasma op

* still not working

* fix

* fix

* comments

* yapf

* silly flake8

* small test
2018-09-19 21:12:37 -07:00
Praveen PalanisamyandEric Liang b23fd5de13 [rllib] Adds agent name & env id to default logdir prefix (#2859)
* Added agent name & env id to default logdir prefix

* Revert "Added agent name & env id to default logdir prefix"

This reverts commit 07cfdf80d2537da3c67dd4f553c5f3e43671cc7d.

* Added default logger creator with informative prefix to Agent

* Updated import order & improved str cat

* Update agent.py
2018-09-18 22:22:07 -07:00
Eric LiangandGitHub 3a3782c39f [rllib] Fix LSTM regression on truncated sequences and add regression test (#2898)
* fix

* add test

* yapf

* yapf

* fix space

* Oops that should be lstm: True

* Update cartpole_lstm.py
2018-09-18 15:09:16 -07:00
Eric LiangandGitHub ab8348b1f5 [rllib] Reward clipping should default to off 2018-09-18 15:08:01 -07:00
Robert NishiharaandHao Chen ea9d1cc887 Remove dependence on psutil. Add utility functions for getting system memory. (#2892) 2018-09-18 15:03:29 +08:00
Robert NishiharaandEric Liang 61bf6c6123 Fix regression in directing worker output to stdout/stderr. (#2897) 2018-09-17 16:40:45 -07:00
Richard LiawandRobert Nishihara 899e4585bc Don't include redundant entries in global_state.client_table (#2880) 2018-09-17 12:52:49 -07:00
Richard LiawandGitHub f372f48bf3 [tune] Tune onto Logging Module (#2882)
Moves Tune onto logging in Python. Ignores examples and tests.
2018-09-16 12:09:36 -07:00
Robert NishiharaandPhilipp Moritz 503344149f Run jupyter UI with --ip=0.0.0.0. (#2883) 2018-09-15 21:59:46 -07:00
Richard LiawandGitHub e05baed336 [tune] Better Info String and Tweaks (#2874) 2018-09-15 11:02:13 -07:00
Hanwei JinandPhilipp Moritz fbf214e408 update ray cmake build process (#2853)
* use cmake to build ray project, no need to appply build.sh before cmake, fix some abuse of cmake, improve the build performance

* support boost external project, avoid using the system or build.sh boost

* keep compatible with build.sh, remove boost and arrow build from it.

* bugfix: parquet bison version control, plasma_java lib install problem

* bugfix: cmake, do not compile plasma java client if no need

* bugfix: component failures test timeout machenism has problem for plasma manager failed case

* bugfix: arrow use lib64 in centos, travis check-git-clang-format-output.sh does not support other branches except master

* revert some fix

* set arrow python executable, fix format error in component_failures_test.py

* make clean arrow python build directory

* update cmake code style, back to support cmake minimum version 3.4
2018-09-12 11:19:33 -07:00
Daniel HoandRichard Liaw d9eeaaf00a [tune] Fix bug in example where config hyperparameters were ignored (#2860)
A fix to an example for tune (`python/ray/tune/examples/pbt_tune_cifar10_with_keras.py`) where the hyperparameters for the optimizer, learning rate and decay, were not being passed into the optimizer. 

This means that the current optimizer uses default values for the hyperparameters no matter the config.
2018-09-12 09:17:56 -07:00
old-bearandRichard Liaw f3c1194be3 [tune] Add AutoML algorithm of GeneticSearcher (#2699)
Add new search algorithm (genetic) along with the base framework of the searcher (which performs some basic jobs such as logging, recording and organizing in our project).
Note that this is the initial commit. In the following days, we will add example, UT, and other refinements.
2018-09-12 09:17:04 -07:00
Eric LiangandGitHub bee743c152 Remove log suppression code
When running in a screen (or any other time it is hard to scroll up), printing "Suppressing previous error message" is not helpful since the previous error is lost far above past scrollback. Better to just print it repeatedly at the end.
 tada 1
2018-09-11 23:28:45 -07:00
KaahanandRichard Liaw 045861c9b0 [tune] Reset Config for Trainables (#2831)
Adds the ability for trainables to reset their configurations during experiments. These changes in particular add the base functions to the trial_executor and trainable interfaces as well as giving the basic implementation on the PopulationBasedTraining scheduler.

Related issue number: #2741
2018-09-11 08:45:04 -07:00
Peter SchafhalterandRobert Nishihara 5da6e78db1 Add available resources to global state (#2501) 2018-09-10 15:46:32 -07:00
Eric LiangandRobert Nishihara 611259b2c7 Re-raise actor initialization errors on method invocation (#2843)
If an actor constructor fails, save that error and re-raise it on any subsequent attempts to interact with the actor. Related to https://github.com/ray-project/ray/issues/282 and https://github.com/ray-project/ray/issues/1093.
2018-09-10 10:51:19 -07:00
Eric LiangandRobert Nishihara 588c573d41 Ray stop needs to kill plasma_store_server not plasma_store (#2850) 2018-09-09 19:23:09 -07:00
eugenevinitskyandEric Liang 9ba751c29a Ars increase (#2844)
* removed cv2

* remove opencv

* increased number of default rollouts ARS

* put cv2 back in this branch

* put cv2 back in this branch

* moved cv2 back where it belongs in preprocessors
2018-09-08 14:09:02 -07:00
Robert NishiharaandPhilipp Moritz bd64c940e9 Push error to driver when monitor raises an exception. (#2834) 2018-09-07 17:42:45 -07:00
Robert NishiharaandPhilipp Moritz 3f6ed537a4 Add ray.is_initialized() function. (#2818)
* Add ray.is_initialized() function.

* Add assert.
2018-09-06 21:20:59 -07:00
Eric LiangandRobert Nishihara e7db54bdb0 Log at INFO level by default (including in autoscaler). (#2824)
Before this change, the autoscaler `up` and related commands don't print any info messages to the console at all. This was a regression from 0.5. @richardliaw @robertnishihara https://github.com/ray-project/ray/issues/2812
2018-09-06 13:31:19 -07:00
Eric LiangandGitHub d81605e9e7 [tune] Add a time/timesteps since last restore metric (#2819)
* rsm

* always log to avoid changing schema for csv writer

* add iter since restore

* update

* criteria warn
2018-09-05 17:45:09 -07:00
Eric LiangandGitHub 995ac24a2c [rllib] clarify train batch size for PPO (#2793)
It's possible to configure PPO in a way that ends up discarding most of the samples (they are treated as "stragglers"). Add a warning when this happens, and raise an exception if the waste is particularly egregious.
2018-09-05 12:06:13 -07:00
karyandEric Liang 4c0e2c3f58 [rllib]multi agent judge bug (#2821)
* fix multi agent judge bug

* Update policy_evaluator.py
2018-09-04 21:02:06 -07:00
Richard LiawandEric Liang 72542c9016 [tune] Fix Pausing and Error Propogation (#2815)
* add new tests

* Try-catch errors from ray get

* longer pbt run

* Update pbt_example.py

* Split trial and result and fix tests
2018-09-04 15:22:11 -07:00
Eric LiangandGitHub 25ffe57a5c [rllib] Auto-synchronize filters for all agents (#2791)
This makes sure we always update the local filter, and adds an option to synchronize the remote filters as well. In APEX_DDPG we previously didn't do either. The first is needed for checkpoint correctness, the second might help performance.
2018-09-03 20:01:53 -07:00
Eric LiangandGitHub 01b030bd57 [rllib] throw an error for continuous action spaces in IMPALA
We currently don't support this since the reference vtrace.py does not, though it could be an interesting extension.
2018-09-03 11:12:55 -07:00
Eric LiangandGitHub df4788e501 [rllib/tune] Add test for fractional gpu support in xray mode; add rllib support for fractional gpu (#2768)
* frac gpu

* doc

* Update rllib-training.rst

* yapf

* remove xray
2018-09-03 11:12:23 -07:00
Eric LiangandRobert Nishihara b37a283053 [rllib] support local mode (#2795) 2018-09-02 23:02:19 -07:00
Robert NishiharaandPhilipp Moritz 0ac855e061 Push errors to all drivers when node is marked dead. (#2808)
* Push errors to all drivers when node is marked dead.

* Fix
2018-09-02 20:04:58 -07:00
Alexey TumanovandRobert Nishihara fdc9688226 [xray] push warning to driver for infeasible tasks (#2784)
This PR pushes a warning to the user for infeasible tasks to alert them to the fact that they can't currently be executed. Fixes #2780.
2018-09-01 13:21:27 -07:00
Robert NishiharaandPhilipp Moritz eda6ebb87d Convert some unittests to pytest. (#2779)
* Convert multi_node_test.py to pytest.

* Convert array_test.py to pytest.

* Convert failure_test.py to pytest.

* Convert microbenchmarks to pytest.

* Convert component_failures_test.py to pytest and some minor quotes changes.

* Convert tensorflow_test.py to pytest.

* Convert actor_test.py to pytest.

* Fix.

* Fix
2018-08-31 11:24:15 -07:00
wangyiguangandRichard Liaw 3813ae34b3 [tune] Add AutoMLBoard: Monitoring UI (experimental) (#2574) 2018-08-31 00:26:44 -07:00
Richard LiawandGitHub 0347e6418b [tune] Add PyTorch MNIST Example + Misc. Tweaks (#2708) 2018-08-30 16:18:56 -07:00
Robert NishiharaandPhilipp Moritz 224d38cbb2 Name Python threads. (#2767) 2018-08-30 11:08:24 -07:00