Commit Graph

130 Commits

Author SHA1 Message Date
Sven Mika c524f86785 [RLlib] BC/MARWIL/recurrent nets minor cleanups and bug fixes. (#13064) 2020-12-27 09:46:03 -05:00
Sven Mika a5318961de [RLlib] Preprocessor fixes (multi-discrete) and tests. (#13083) 2020-12-26 20:14:36 -05:00
Sven Mika 99ae7bae05 [RLlib] JAXPolicy prep. PR #1. (#13077) 2020-12-26 20:14:18 -05:00
Sven Mika d5604eaba3 [RLlib] Attention nets PyTorch support and cleanup (using traj. view API). (#12029) 2020-12-21 18:38:34 -08:00
Sven Mika b2bcab711d [RLlib] Attention Nets: tf (#12753) 2020-12-20 20:22:32 -05:00
Sven Mika 124c8318a8 [RLlib] Fix broken test_distributions.py (test_categorical) (#12915) 2020-12-17 17:44:26 -06:00
Sven Mika abb1eefdc2 [RLlib] Issue 12483: Discrete observation space error: "ValueError: ('Observation ({}) outside given space ..." when doing Trainer.compute_action. (#12787) 2020-12-11 22:43:30 +01:00
Sven Mika 99c81c6795 [RLlib] Attention Net prep PR #3. (#12450) 2020-12-07 13:08:17 +01:00
Sven Mika 3f4bc16276 [RLlib] Add a minimal JAX ModelV2 (FCNet) to RLlib. (#12502) 2020-12-03 15:51:30 +01:00
Sven Mika 19c8033df2 [RLlib] Fix most remaining RLlib algos for running with trajectory view API. (#12366)
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* LINT and fixes.
MB-MPO and MAML not working yet.

* wip

* update

* update

* rmeove

* remove dep

* higher

* Update requirements_rllib.txt

* Update requirements_rllib.txt

* relpos

* no mbmpo

Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-12-01 17:41:10 -08:00
Sven Mika 3ad9365e1d [RLlib] Attention Net prep PR #2: Smaller cleanups. (#12449) 2020-12-01 08:21:45 +01:00
Sven Mika 592c161032 [RLlib] Issue 12118: LSTM prev-a/r should be separately configurable. Fix missing prev-a one-hot encoding. (#12397)
* WIP.

* Fix and LINT.
2020-11-25 11:27:46 -08:00
Sven Mika 62c7ab5182 [RLlib] Trajectory view API: Enable by default for PPO, IMPALA, PG, A3C (tf and torch). (#11747) 2020-11-12 16:27:34 +01:00
Michael Luo 59ccbc0fc7 [RLlib] Model Annotations: Tensorflow (#11964) 2020-11-12 12:18:50 +01:00
Michael Luo b2984d1c34 [RLlib] Model Annotations to Torch Models (#9749) 2020-11-12 12:16:12 +01:00
Sven Mika 291c172d83 [RLlib] Support Simplex action spaces for SAC (torch and tf). (#11909) 2020-11-11 18:45:28 +01:00
Sven Mika 5b788ccb13 [RLlib] Trajectory view API (prep PR for switching on by default across all RLlib; plumbing only) (#11717) 2020-11-03 12:53:34 -08:00
Jiajie Xiao 0b07af374a allow tuple action space (#11429)
Co-authored-by: Jiajie Xiao <jj@Jiajies-MBP-2.attlocal.net>
2020-10-29 16:05:38 +01:00
Sven Mika d9f1874e34 [RLlib] Minor fixes (torch GPU bugs + some cleanup). (#11609) 2020-10-27 10:00:24 +01:00
Sven Mika 1ebcdf236f [RLlib] Add support for custom MultiActionDistributions. (#11311) 2020-10-12 13:50:43 -07:00
Sven Mika 0c0f67c14d [RLlib] ARS/ES eval workers not working: Issue 9933. (#11308) 2020-10-12 13:49:48 -07:00
Sven Mika d3bc20b727 [RLlib] ConvTranspose2D module (#11231) 2020-10-12 15:00:42 +02:00
Sven Mika 957877ad3f Tf version of VisionNet (ray/rllib/model/tf/vision_net.py) crashes iff len(conv-filters)=1. (#11330) 2020-10-11 12:49:47 +02:00
Sumanth Ratna 14d8826e43 Fix overriden typo (#11227) 2020-10-07 19:11:07 -07:00
Sven Mika ce96b03b07 [RLlib] MB-MPO cleanup (comments, docstrings, type annotations). (#11033) 2020-10-06 20:28:16 +02:00
Sven Mika c17169dc11 [RLlib] Fix all example scripts to run on GPUs. (#11105) 2020-10-02 23:07:44 +02:00
Sven Mika 36bda8432b [RLlib] Trajectory view API: Simple List Collector (on by default for PPO); LSTM-agnostic (#11056) 2020-10-01 16:57:10 +02:00
internetcoffeephone 840fb5543b Change get_action_shape so that it uses the dtype of the Discrete object, rather than overwriting it with tf.int64. (#8424) 2020-09-21 17:08:31 -07:00
Sven Mika 805dad3bc4 [RLlib] SAC algo cleanup. (#10825) 2020-09-20 11:27:02 +02:00
Sven Mika 28ab797cf5 [RLlib] Deprecate old classes, methods, functions, config keys (in prep for RLlib 1.0). (#10544) 2020-09-06 10:58:00 +02:00
Sven Mika 8a891b3c30 [RLlib] SAC n_step > 1. (#10567) 2020-09-05 22:26:42 +02:00
Sven Mika 244aafdcf8 [RLlib] Curiosity enhancements. (#10373) 2020-09-05 13:14:24 +02:00
Sven Mika ef18893fb5 [RLlib] PPO, APPO, and DD-PPO code cleanup. (#10420) 2020-09-02 14:03:01 +02:00
Sven Mika e968b52cb7 [RLlib] Trajectory view API - 03 Fast LSTM + prev actions/rewards (#9950) 2020-08-21 12:35:16 +02:00
Sven Mika 2cbe29a7fa [RLlib] Curiosity minor fixes, do-overs, and testing. (#10143) 2020-08-19 17:49:50 +02:00
Eric Liang ca133e2699 [rllib] Remove extra model config kwargs passed incorrectly for Torch models (#10055) 2020-08-17 11:12:20 -07:00
Sven Mika 2256047876 [RLlib] Rename rllib.utils.types into typing to match built-in python module's name. (#10114) 2020-08-15 13:24:22 +02:00
Sven Mika 0effcda3e4 Add missing int-casts for all shape calculating code (using np.product([some shape])). (#10092) 2020-08-13 12:04:22 +02:00
Barak Michener 8e76796fd0 ci: Redo format.sh --all script & backfill lint fixes (#9956) 2020-08-07 16:49:49 -07:00
Sven Mika 5d5643e633 [RLlib] Add informative error message when bad Conv2D stack is used with fixed num_outputs (no flattening at end). (#9966) 2020-08-07 12:04:17 +02:00
Sven Mika 57690a3a9f [RLlib] Trajectory view API - 02 actual API scaffold (#9753) 2020-08-06 10:54:20 +02:00
Sven Mika 9b90f7db67 [RLlib] Missing type annotations policy templates. (#9846) 2020-08-06 05:33:24 +02:00
Sven Mika b0b0463161 [RLlib] Trajectory View API (preparatory cleanup and enhancements). (#9678) 2020-07-29 21:15:09 +02:00
Sven Mika 5dc4b6686e [RLlib] Implement DQN PyTorch distributional head. (#9589) 2020-07-25 09:29:24 +02:00
Eric Liang 590943a499 [rllib] Type annotations for model classes (#9646) 2020-07-24 12:01:46 -07:00
Raphael Avalos 5303c3abe3 Fix TorchDeterministic (#9241) 2020-07-23 10:43:20 -07:00
Sven Mika 935d8308fb [RLlib] Issue #9437 (PyTorch converts to CPU tensor, even if on GPU). (#9497) 2020-07-16 14:55:50 +02:00
Sven Mika 03ab86567f [RLlib] Layout of Trajectory View API (new class: Trajectory; not used yet). (#9269) 2020-07-14 04:27:49 +02:00
Sven Mika fcdf410ae1 [RLlib] Tf2.x native. (#8752) 2020-07-11 22:06:35 +02:00
Sven Mika 01125b8fcf [RLlib] DQN rainbow eager-mode (keras style NoisyLayer) (preparation for native tf2.x support). (#9304) 2020-07-09 10:44:10 +02:00