wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-07-02 08:46:59 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,317 Commits 36 Branches 42 Tags
a96ddec3582e1f1007f41c4bbaf31761ba3851d4
Commit Graph

82 Commits

Author SHA1 Message Date
Sven Mika b0b0463161 [RLlib] Trajectory View API (preparatory cleanup and enhancements). (#9678) 2020-07-29 21:15:09 +02:00
Sven Mika e6ea33a03c [RLlib] Enhance reward clipping test; add action_clipping tests. (#9684) 2020-07-28 10:44:54 +02:00
Sven Mika 5dc4b6686e [RLlib] Implement DQN PyTorch distributional head. (#9589) 2020-07-25 09:29:24 +02:00
Sven Mika 78dfed2683 [RLlib] Issue 8384: QMIX doesn't learn anything. (#9527) 2020-07-17 12:14:34 +02:00
Sven Mika 935d8308fb [RLlib] Issue #9437 (PyTorch converts to CPU tensor, even if on GPU). (#9497) 2020-07-16 14:55:50 +02:00
Sven Mika 617eb8f279 [RLlib] Issue 9402 MARWIL producing nan rewards. (#9429) 2020-07-14 05:07:16 +02:00
Sven Mika 14160ca58c [RLlib] Issue #9366 (DQN w/o dueling produces invalid actions). (#9386) 2020-07-10 12:43:03 +02:00
Benjamin Black 1425cdf834 Pettingzoo environment support (#9271) 
* added pettingzoo wrapper env and example

* added docs, examples for pettingzoo env support

* fixed pettingzoo env flake8, added test

* fixed pettingzoo env import

* fixed pettingzoo env import

* fixed pettingzoo import issue

* fixed pettingzoo test

* fixed linting problem

* fixed bad quotes

* future proofed pettingzoo dependency

* fixed ray init in pettingzoo env

* lint

* manual lint

Co-authored-by: Eric Liang <ekhliang@gmail.com>
 2020-07-06 21:32:26 -07:00
Sven Mika f43d934817 [RLlib] Type annotations for policy. (#9248) 2020-07-05 13:09:51 +02:00
Michael Luo 851d02463b [Doc] RLlib Algorithms Documentation: MAML + PyTorch MAML (#9189) 2020-07-03 11:05:15 -07:00
Sven Mika 5b2a97597b [RLlib] Retire try_import_tree (should be installed along with other requirements). (#9211) 
- Retire try_import_tree.
- Stabilize test_supported_multi_agent.py.
 2020-07-02 13:06:34 +02:00
Sven Mika 43043ee4d5 [RLlib] Tf2x preparation; part 2 (upgrading try_import_tf()). (#9136) 
* WIP.

* Fixes.

* LINT.

* WIP.

* WIP.

* Fixes.

* Fixes.

* Fixes.

* Fixes.

* WIP.

* Fixes.

* Test

* Fix.

* Fixes and LINT.

* Fixes and LINT.

* LINT.
 2020-06-30 10:13:20 +02:00
Sven Mika 5c6d5d4ab1 This PR fixes the currently broken lstm_use_prev_action_reward flag for default lstm models (model.use_lstm=True). (#8970) 2020-06-27 20:50:01 +02:00
Sven Mika af1203b9df [RLlib] Issue 8507 (PyTorch does not support custom loss). (#9142) 2020-06-26 09:52:22 +02:00
Sven Mika 4fd8977eaf [RLlib] Minor cleanup in preparation to tf2.x support. (#9130) 
* WIP.

* Fixes.

* LINT.

* Fixes.

* Fixes and LINT.

* WIP.
 2020-06-25 19:01:32 +02:00
Michael Luo cf0894d396 [rllib] MAML Agent (#8862) 
* Halfway done with transferring MAML to new Ray

* MAML Beta Out

* Debugging MAML atm

* Distributed Execution

* Pendulum Mass Working

* All experiments complete

* Cleaned up codebase

* Travis CI

* Travis CI

* Tests

* Merged conflicts

* Fixed variance bug conflict

* Comment resolved

* Apply suggestions from code review

fixed test_maml

* Update rllib/agents/maml/tests/test_maml.py

* asdf

* Fix testing

Co-authored-by: Sven Mika <sven@anyscale.io>
 2020-06-23 09:48:23 -07:00
Sven Mika 14405b90d5 [RLlib] Prototype of a DynaTrainer (for env dynamics learning in upcoming MBMPO algo). (#8860) 2020-06-16 09:01:20 +02:00
Sven Mika 7008902cff [RLlib] Minor rllib.utils cleanup. (#8932) 2020-06-16 08:52:20 +02:00
Sven Mika a90cd0fcbb [RLlib] Unity3d soccer benchmarks (#8834) 2020-06-11 14:29:57 +02:00
Sven Mika 0ba7472da9 [Testing] Fix LINT/sphinx errors. (#8874) 2020-06-10 15:41:59 +02:00
Eric Liang be26a7b1b0 [rllib] Support for complex / variable-length observation spaces (#8393) 2020-06-06 12:22:19 +02:00
Sven Mika c74dc58f8b [RLlib] Fix use_lstm flag for ModelV2 (w/o ModelV1 wrapping) and add it for PyTorch. (#8734) 2020-06-05 15:40:30 +02:00
Sven Mika d8a081a185 [RLlib] Unity3D integration (n Unity3D clients vs learning server). (#8590) 2020-05-30 22:48:34 +02:00
Sven Mika 2746fc0476 [RLlib] Auto-framework, retire use_pytorch in favor of framework=... (#8520) 2020-05-27 16:19:13 +02:00
Sven Mika 6d196197bc [RLlib] utils/spaces ... (#8608) 2020-05-27 10:21:30 +02:00
Sven Mika 0422e9c5a8 [RLlib] Add 2 Transformer learning test cases on StatelessCartPole (PPO and IMPALA). (#8624) 2020-05-27 10:19:47 +02:00
Eric Liang 9a83908c46 [rllib] Deprecate policy optimizers (#8345) 2020-05-21 10:16:18 -07:00
Sven Mika 796a834c48 [RLlib] Attention Net integration into ModelV2 and learning RL example. (#8371) 2020-05-18 17:26:40 +02:00
Sven Mika 5f4c196fed [RLlib] Make PyTorch Model forward pass faster in vf-case. (#8422) 2020-05-14 10:15:50 +02:00
Sven Mika 57544b1ff9 [RLlib] Examples folder restructuring (Model examples; final part). (#8278) 
- This PR completes any previously missing PyTorch Model counterparts to TFModels in examples/models.
- It also makes sure, all example scripts in the rllib/examples folder are tested for both frameworks and learn the given task (this is often currently not checked) using a --as-test flag in connection with a --stop-reward.
 2020-05-12 08:23:10 +02:00
Eric Liang 9d012626e5 [rllib] Distributed exec workflow for impala (#8321) 2020-05-11 20:24:43 -07:00
Sven Mika d946f58fd0 LINT fixes. (#8370) 2020-05-08 16:24:20 +02:00
gehring 7f14fb577d [RLlib] Added TransformerXL and "stabilized for RL" variant, GTrXL (#6470) 2020-05-08 14:10:23 +02:00
Eric Liang 9f04a65922 [rllib] Add PPO+DQN two trainer multiagent workflow example (#8334) 2020-05-07 23:40:29 -07:00
Sven Mika d7eaacb5fe [RLlib] Issue 8319 DDPG (MA or num_envs_per_worker > 1) broken. (#8324) 2020-05-08 08:26:32 +02:00
Sven Mika 5f278c6411 [RLlib] Examples folder restructuring (models) part 1 (#8353) 2020-05-08 08:20:18 +02:00
Eric Liang 30db920787 [rllib] Fix centralized critic example to use right policy (#8341) 
* update

* update
 2020-05-07 10:47:55 -07:00
Eric Liang f48da50e1c [rllib] observation function api for multi-agent (#8236) 2020-05-04 22:13:49 -07:00
Sven Mika 42991d723f [RLlib] rllib/examples folder restructuring (#8250) 
Cleans up of the rllib/examples folder by moving all example Envs into rllibexamples/env (so they can be used by other scripts and tests as well).
 2020-05-01 22:59:34 +02:00
Sven Mika bf25aee392 [RLlib] Deprecate all Model(v1) usage. (#8146) 
Deprecate all Model(v1) usage.
 2020-04-29 12:12:59 +02:00
Sven Mika 1775e89f26 [RLlib] Remove TupleActions and support arbitrarily nested action spaces. (#8143) 
Deprecate TupleActions and support arbitrarily nested action spaces.
Closes issue #8143.
 2020-04-28 14:59:16 +02:00
Eric Liang 17e3c545d9 [rllib] Fix truncate episodes mode in central critic example (#8073) 2020-04-20 12:58:01 -07:00
roireshef dbcad35022 [RLlib] Added DefaultCallbacks which replaces old callbacks dict interface (#6972) 2020-04-16 16:06:42 -07:00
Sven Mika 22ccc43670 [RLlib] DQN torch version. (#7597) 
* Fix.

* Rollback.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Fix.

* Fix.

* Fix.

* Fix.

* Fix.

* WIP.

* WIP.

* Fix.

* Test case fixes.

* Test case fixes and LINT.

* Test case fixes and LINT.

* Rollback.

* WIP.

* WIP.

* Test case fixes.

* Fix.

* Fix.

* Fix.

* Add regression test for DQN w/ param noise.

* Fixes and LINT.

* Fixes and LINT.

* Fixes and LINT.

* Fixes and LINT.

* Fixes and LINT.

* Comment

* Regression test case.

* WIP.

* WIP.

* LINT.

* LINT.

* WIP.

* Fix.

* Fix.

* Fix.

* LINT.

* Fix (SAC does currently not support eager).

* Fix.

* WIP.

* LINT.

* Update rllib/evaluation/sampler.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/evaluation/sampler.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/utils/exploration/exploration.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/utils/exploration/exploration.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* WIP.

* WIP.

* Fix.

* LINT.

* LINT.

* Fix and LINT.

* WIP.

* WIP.

* WIP.

* WIP.

* Fix.

* LINT.

* Fix.

* Fix and LINT.

* Update rllib/utils/exploration/exploration.py

* Update rllib/policy/dynamic_tf_policy.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/policy/dynamic_tf_policy.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/policy/dynamic_tf_policy.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Fixes.

* WIP.

* LINT.

* Fixes and LINT.

* LINT and fixes.

* LINT.

* Move action_dist back into torch extra_action_out_fn and LINT.

* Working SimpleQ learning cartpole on both torch AND tf.

* Working Rainbow learning cartpole on tf.

* Working Rainbow learning cartpole on tf.

* WIP.

* LINT.

* LINT.

* Update docs and add torch to APEX test.

* LINT.

* Fix.

* LINT.

* Fix.

* Fix.

* Fix and docstrings.

* Fix broken RLlib tests in master.

* Split BAZEL learning tests into cartpole and pendulum (reached the 60min barrier).

* Fix error_outputs option in BAZEL for RLlib regression tests.

* Fix.

* Tune param-noise tests.

* LINT.

* Fix.

* Fix.

* test

* test

* test

* Fix.

* Fix.

* WIP.

* WIP.

* WIP.

* WIP.

* LINT.

* WIP.

Co-authored-by: Eric Liang <ekhliang@gmail.com>
 2020-04-06 11:56:16 -07:00
Sven Mika e153e3179f [RLlib] Exploration API: Policy changes needed for forward pass noisifications. (#7798) 
* Rollback.

* WIP.

* WIP.

* LINT.

* WIP.

* Fix.

* Fix.

* Fix.

* LINT.

* Fix (SAC does currently not support eager).

* Fix.

* WIP.

* LINT.

* Update rllib/evaluation/sampler.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/evaluation/sampler.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/utils/exploration/exploration.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/utils/exploration/exploration.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* WIP.

* WIP.

* Fix.

* LINT.

* LINT.

* Fix and LINT.

* WIP.

* WIP.

* WIP.

* WIP.

* Fix.

* LINT.

* Fix.

* Fix and LINT.

* Update rllib/utils/exploration/exploration.py

* Update rllib/policy/dynamic_tf_policy.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/policy/dynamic_tf_policy.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/policy/dynamic_tf_policy.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Fixes.

* LINT.

* WIP.

Co-authored-by: Eric Liang <ekhliang@gmail.com>
 2020-04-01 00:43:21 -07:00
Sven Mika 66df8b8c35 [RLlib] Working/learning example: PPO + torch + LSTM. (#7797) 2020-03-31 22:00:28 -07:00
Sven Mika e356e97eb2 [RLlib] Assert correct policy class being used in Worker. (#7769) 2020-03-30 14:03:29 -07:00
Eric Liang 9a590ac6a5 [rllib] Fix custom model metrics in multi-device case (#7640) 
* fix example

* add example test

* lin
 2020-03-23 12:40:22 -07:00
Eric Liang 9392cdbf74 [rllib] Add high-performance external application connector (#7641) 2020-03-20 12:43:57 -07:00
mehrdadn a0700e2f86 Change /tmp to platform-specific temporary directory (#7529) 2020-03-16 18:10:14 -07:00