wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-09-11 12:43:20 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,987 Commits 36 Branches 42 Tags
445bcb29b0bd586a58f9ef2190c25946e67913e9
Commit Graph
465 Commits
Author SHA1 Message Date
Eric Liang d9da183c7d [rllib] Custom supervised loss API (#4083) 2019-02-24 15:36:13 -08:00
Robert Nishihara 7b04ed059e Move TensorFlowVariables to ray.experimental.tf_utils. (#4145) 2019-02-24 14:26:46 -08:00
Eric Liang 05d96ce81b [rllib] Raise an error if multi-agent envs terminate without a last observation for agents (#4139) 
* fix it

* lint

* Update rllib-training.rst
 2019-02-23 21:23:40 -08:00
Eric Liang 9896df7799 [rllib] Guard against PPO value function not training with RNN models (#4037) 
* better lstm settings

* 1.0

* docs

* warn on truncate

* clarify

* Update ppo_policy_graph.py

* Update ppo_policy_graph.py

* Update ppo_policy_graph.py
 2019-02-22 11:18:51 -08:00
Stefan Pantic a54386e499 Added custom LSTM detection (#4087) 
* Added autodetection of custom LSTM usage

* Reverted line separators

* Added check for LSTM

* Update vtrace_policy_graph.py

* Update appo_policy_graph.py
 2019-02-21 21:07:48 -08:00
William Ma c7a4c74f55 Moving tests from test/ to python/ray/tests/ (#3950) 2019-02-21 11:09:08 -08:00
Jones Wong acbe0b4e5f Fix twin q bug (#4108) 2019-02-21 10:47:01 -08:00
Jones Wong 3ac8fd7ee8 Exploration with Parameter Space Noise (#4048) 
*  enable parameter space noise for exploration

*  enable parameter space noise for exploration

*  yapf formatted

*  remove the usage of scipy softmax avialable in the latest version only

*  enable subclass that has no parameter_noise in the config

*  run user specified callbacks and test parameter space noise in multi node setting

*  formatted by yapf

* Update dqn.py

* lint
 2019-02-20 22:35:18 -08:00
Richard Liaw acf4d53b55 [autoscaler] Fix redirects, fix submit (#4085) 2019-02-20 21:35:33 -08:00
mika 64c95aea85 [rllib] Update README.md for qmix (#4101) 
## What do these changes do?

Fixed PyMARL repository path.

## Related issue number

N/A
 2019-02-20 10:21:08 -08:00
Eric Liang f8bef004da [rllib] Improve error message for bad envs, add remote env docs (#4044) 
* commit

* fix up rew
 2019-02-18 01:28:19 -08:00
Philipp Moritz f51969964d Fix linting on master (#4077) 2019-02-17 13:55:40 -08:00
Megan Kawakami 346885068c [rllib] add torch pg (#3857) 
* add torch pg

* add torch imports

* added torch pg

* working torch pg implementation

* add pg pytorch

* Update a3c.py

* Update a3c.py

* Update torch_policy_graph.py

* Update torch_policy_graph.py
 2019-02-16 19:54:14 -08:00
Zekun Shi a708ab66f5 Add simplex action space and dirichlet action distribution (#4070) 
* add simplex action space and dirichlet action distribution

* Update and rename spaces.py to extra_spaces.py

* Update __init__.py

* Update catalog.py

* Fix python 2

* Update extra_spaces.py

* change Simplex.contains() to return False
 2019-02-16 12:44:59 -08:00
Robert Nishihara 2d07df7f3f Replace '__main__' with "__main__". (#4055) 2019-02-15 13:32:43 -08:00
Robert Nishihara 5f71751891 API cleanups. Remove worker argument. Remove some deprecated arguments. (#4025) 
* Remove worker argument from API methods.

* Remove deprecated arguments and deprecate redirect_output and redirect_worker_output.

* Fix
 2019-02-15 10:49:16 -08:00
Eric Liang 0c0bd4d41c [rllib] Use model.value_function() in MARWIL (#4036) 
* fix marwil

* add ph

* fix
 2019-02-14 19:35:21 -08:00
Eric Liang 2dccf383dd [rllib] Basic infrastructure for off-policy estimation (IS, WIS) (#3941) 2019-02-13 16:25:05 -08:00
bjg2 0e37ac6d1d [wingman -> rllib] Remote and entangled environments (#3968) 
* added all our environment changes

* fixed merge request comments and remote env

* fixed remote check

* moved remote_worker_envs to correct config section

* lint

* auto wrap impl

* fix

* fixed the tests
 2019-02-13 10:08:26 -08:00
Eric Liang 8df772867c [rllib] rename compute_apply to learn_on_batch 2019-02-11 15:22:15 -08:00
Eric Liang c4182463f6 [rllib] Add helper to iterate over envs in a vectorized environment (#4001) 
* add foreach env func

* fix

* add test
 2019-02-11 10:40:47 -08:00
bjg2 e703b9f49d [wingman -> rllib] Improved stats changes in AsyncSamplesOptimizer (#3966) 
* added stats changes to optimizer

* changes timers

* fix python 2 compat

* improved optimizer throughput stats

* Update async_samples_optimizer.py

* fix python2 compat
 2019-02-10 01:25:22 -08:00
Eric Liang 29322c7389 [rllib] Replay buffer for IMPALA should default to 0 slots. (#3971) 
* disable replay

* make lq configurable

* leak test

* Update run_multi_node_tests.sh
 2019-02-08 10:03:11 -08:00
Eric Liang ae4bc7d6e8 [revert] [rllib] Add copy() in async samples optimizer 2019-02-07 14:14:39 -08:00
Eric Liang 04fc145a44 [autoscaler] Autoscaler hangs forever on non-zero exit code command (#3969) 2019-02-06 17:25:24 -08:00
Eric Liang 7ef830bef1 [rllib] Add copy() in async samples optimizer to fix memory leak (#3938) 
Fixes #3884.
 2019-02-03 18:34:37 -08:00
Michael Luo 1a015e420b Optimal PPO Configs (10k reward in 1 hr) + PPO grad clipping implemented (#3934) 2019-02-02 22:10:58 -08:00
Eric Liang 0f81bc9a33 [rllib] on_train_result results do not get logged (#3865) 2019-02-01 20:32:07 -08:00
Tianming Xu 1302fafc0b [Tune] Add export_formats option to export policy graphs (#3868) 
In earlier PRs, PR#3585 and PR#3637, export_policy_model and export_policy_checkpoint were introduced for users to export TensorFlow model and checkpoint.

For Ray Tune users, these APIs are not accessible through YAML configurations.

In this pull request, export_formats option is provided to enable users to choose the desired export format.
 2019-01-31 17:07:27 -08:00
Eric Liang 152375aa8a [rllib] Add evaluation option to DQN agent (#3835) 
* add eval

* interval

* multiagent minor fix

* Update rllib.rst

* Update ddpg.py

* Update qmix.py
 2019-01-29 21:19:53 -08:00
Eric Liang fb73cedf70 [rllib] Add examples page, add hierarchical training example, delete SC2 examples (#3815) 
* wip

* lint

* wip

* up

* wip

* update examples

* wip

* remove carla

* update

* improve envspec

* link to custom

* Update rllib-env.rst

* update

* fix

* fn

* lint

* ds

* ssd games

* desc

* fix up docs

* fix
 2019-01-29 21:06:09 -08:00
Si-Yuan 48139cf861 Migrate Python C extension to Cython (#3541) 2019-01-24 09:17:14 -08:00
Eric Liang 04ec47cbd4 [rllib] annotate public vs developer vs private APIs (#3808) 2019-01-23 21:27:26 -08:00
Eric Liang f0e6523323 [rllib] Don't call reset() unless necessary for multi-agent envs 2019-01-20 15:00:18 -08:00
Eric Liang aad48ee5a5 [tune] Fully deprecate raw function literals in Tune (#3788) 
Related: https://github.com/ray-project/ray/issues/3785
 2019-01-19 17:09:36 -08:00
Michael Luo 16f7ca45e4 Appo (#3779) 
* Deleted old fork, updated new ray and moved PPO-impala to APPO in ppo folder

* Deleted unneccesary vtrace.py file

* Update pong-impala.yaml

* Cleaned PPO Code

* Update pong-impala.yaml

* Update pong-impala.yaml

* wip

* new ifle

* refactor

* add vtrace off option

* revert

* support any space

* docs

* fix comment

* remove kl

* Update cartpole-appo-vtrace.yaml
 2019-01-18 13:40:26 -08:00
Jones Wong 319c1340cb [rllib] Develop MARWIL (#3635) 
*  add marvil policy graph

*  fix typo

*  add offline optimizer and enable running marwil

*  fix loss function

*  add maintaining the moving average of advantage norm

*  use sync replay optimizer for unifying

*  remove offline optimizer and use sync replay optimizer

*  format by yapf

*  add imitation learning objective

*  fix according to eric's review

*  format by yapf

* revise

* add test data

* marwil
 2019-01-16 19:00:43 -08:00
Richard Liaw fa99fda2b4 Application Stress Tests (#3612) 2019-01-16 02:05:16 -08:00
Eric Liang 401e656b95 [rllib] Sync filters at end of iteration not start; hierarchical docs (#3769) 2019-01-15 16:25:25 -08:00
Eugene Vinitsky a5d1f03515 [rllib] fix for rollout of lstm policies (#3643) 
* fix for lstm policies

* added call to local evaluator

* Update python/ray/rllib/rollout.py

Co-Authored-By: eugenevinitsky <eugenevinitsky@users.noreply.github.com>

* Update rollout.py

* Update rollout.py
 2019-01-13 15:54:23 -08:00
Eric Liang 71243203a4 [rllib] Fix KeyError: 'kl' in multiagent ppo training 2019-01-09 19:33:07 -08:00
Robert Nishihara d1e21b702e Change timeout from milliseconds to seconds in ray.wait. (#3706) 
* Change timeout from milliseconds to seconds in ray.wait.

* Suppress warning.

* Suppress warning.

* Add prominent warning in API documentation.
 2019-01-08 21:32:08 -08:00
Robert Nishihara c9d70f0dda Remove num_local_schedulers argument from ray.worker._init. (#3704) 
* Remove num_local_schedulers argument from ray.worker._init.

* Fix

* Fix tests.
 2019-01-07 12:44:49 -08:00
Eric Liang e78562b2e8 [rllib] Misc fixes: set lr for PG, better error message for LSTM/PPO, fix multi-agent/APEX (#3697) 
* fix

* update test

* better error

* compute

* eps fix

* add get_policy() api

* Update agent.py

* better err msg

* fix

* pass in rew
 2019-01-06 19:37:35 -08:00
Eric Liang 03fe760616 [rllib] Model self loss isn't included in all algorithms (#3679) 2019-01-04 22:30:35 -08:00
Eric Liang 7db1f3be2a [tune] resume=False by default but print a tip to set resume="prompt" + jenkins fix (#3681) 2019-01-04 17:23:19 -08:00
Eric Liang ca864faece [rllib] Documentation for I/O API and multi-agent support / cleanup (#3650) 2019-01-03 15:15:36 +08:00
opherlieber 2177e2f410 [rllib] Agent: Allow unknown subkeys for custom_resources_per_worker (#3639) 
* RLLib Agent: Allow unknown subkeys for custom_resources_per_worker

* Update agent.py
 2019-01-03 14:19:59 +08:00
Eric Liang 47d36d7bd6 [rllib] Refactor pytorch custom model support (#3634) 2019-01-03 13:48:33 +08:00
Richard Liaw aad3c50e2d [tune] Cluster Fault Tolerance (#3309) 
This PR introduces cluster-level fault tolerance for Tune by checkpointing global state. This occurs with relatively high frequency and allows users to easily resume experiments when the cluster crashes.

Note that this PR may affect automated workflows due to auto-prompting, but this is resolvable.
 2018-12-29 11:42:25 +08:00