Eric Liang and GitHub
59901a88a0
[rllib] Native support for Dict and Tuple spaces; fix Tuple action spaces; add prev a, r to LSTM ( #3051 )
2018-10-20 15:21:22 -07:00
Eric Liang and GitHub
a9e454f6fd
[rllib] Include config dicts in the sphinx docs ( #3064 )
2018-10-16 15:55:11 -07:00
Eric Liang and GitHub
e4bea8d10e
[rllib] Default to truncate_episodes and add some more config validators ( #2967 )
...
* update
* link it
* warn about truncation
* fix
* Update rllib-training.rst
* deprecate tests failing
2018-09-30 18:37:55 -07:00
Eric Liang and GitHub
65dcafdc3f
[rllib] Refactor save() / restore() code of agents and avoid O(n_workers) save size ( #2982 )
2018-09-30 01:15:13 -07:00
Eric Liang and GitHub
8331d1ebe0
[rllib] Add vf clipping param to fix pendulum example ( #2921 )
...
* add vf clip
* fix test
* Update ppo.py
2018-09-23 13:11:17 -07:00
Eric Liang and GitHub
995ac24a2c
[rllib] clarify train batch size for PPO ( #2793 )
...
It's possible to configure PPO in a way that ends up discarding most of the samples (they are treated as "stragglers"). Add a warning when this happens, and raise an exception if the waste is particularly egregious.
2018-09-05 12:06:13 -07:00
Eric Liang and GitHub
25ffe57a5c
[rllib] Auto-synchronize filters for all agents ( #2791 )
...
This makes sure we always update the local filter, and adds an option to synchronize the remote filters as well. In APEX_DDPG we previously didn't do either. The first is needed for checkpoint correctness, the second might help performance.
2018-09-03 20:01:53 -07:00
Eric Liang and GitHub
aa014af85b
[rllib] Fix atari reward calculations, add LR annealing, explained var stat for A2C / impala ( #2700 )
...
Changes needed to reproduce Atari plots in IMPALA / A2C: https://github.com/ray-project/rl-experiments
2018-08-23 17:49:10 -07:00
Richard Liaw and GitHub
bb44456f6f
[rllib, tune] TrainingResult -> Dict, Removes C408 from flake8 ( #2565 )
2018-08-07 12:17:44 -07:00
Eric Liang and GitHub
981d9818c1
[rllib] Support the timesteps_per_batch in simple optimizer PPO mode ( #2558 )
...
* support ts
* doc
* Update sync_samples_optimizer.py
2018-08-06 12:10:59 -07:00
Eric Liang and GitHub
a630e332f3
[rllib] Don't use get_gpu_ids() in ppo
...
This lets the num_gpus config work properly even when not using tune, since the gpu ids won't be set by ray in that case.
2018-08-01 16:25:11 -07:00
Eric Liang and GitHub
38d00986a5
[rllib] Cleanups: deep merge configs properly; enforce min iter time on APEX ( #2500 )
...
The dict merge prevents crashes when tune is trying to get resource requests for agents and you override a config subkey. The min iter time prevents iterations from getting too small, incurring high overhead. This is easy to run into on Ape-X since throughput can get very high.
2018-07-30 13:25:35 -07:00
Eric Liang and GitHub
68660453e4
[rllib] Better support and add two-trainer example for multiagent ( #2443 )
...
This adds a simple DQN+PPO example for multi-agent. We don't do anything fancy here, just syncing weights between two separate trainers. This potentially is wasting some compute, but is very simple to set up.
It might be nice to share experience collection between the top-level trainers in the future.
2018-07-22 05:09:25 -07:00
Eric Liang and GitHub
d01dc9e22d
[rllib] format with yapf ( #2427 )
...
* initial yapf
* manual fix yapf bugs
2018-07-19 15:30:36 -07:00
Eric Liang and GitHub
0cecf6b79c
[rllib] Cleanup RNN support and make it work with multi-GPU optimizer ( #2394 )
...
Cleanup: TFPolicyGraph now automatically adds loss input entries for state_in_*, so that graph sub-classes don't need to worry about it.
Multi-GPU support:
Allow setting up model tower replicas with existing state input tensors
Truncate the per-device minibatch slices so that they are always a multiple of max_seq_len.
2018-07-17 06:55:46 +02:00
Eric Liang and Richard Liaw
b316afeb43
[rllib] Add debug info back to PPO and fix optimizer compatibility ( #2366 )
2018-07-12 19:22:46 +02:00
Eric Liang and GitHub
d24f19fd1e
[rllib] Fix stats collection and some docs bugs since the refactoring ( #2361 )
...
* fix
* fix pbt example
* fix
* fix
* single thread by default
* vec
* fix
* fix
2018-07-07 13:29:20 -07:00
Richard Liaw and GitHub
e32aed8717
[rllib] more user-friendly Optimizer signature + compute_apply ( #2335 )
...
* Move signature of optimizers
* fix
* expose compute_apply for policy_graphs
* dictionaries and such
* test for multiagent
2018-07-07 12:08:49 -07:00
Eric Liang and GitHub
8aa56c12e6
[rllib] Document "v2" APIs ( #2316 )
...
* re
* wip
* wip
* a3c working
* torch support
* pg works
* lint
* rm v2
* consumer id
* clean up pg
* clean up more
* fix python 2.7
* tf session management
* docs
* dqn wip
* fix compile
* dqn
* apex runs
* up
* impotrs
* ddpg
* quotes
* fix tests
* fix last r
* fix tests
* lint
* pass checkpoint restore
* kwar
* nits
* policy graph
* fix yapf
* com
* class
* pyt
* vectorization
* update
* test cpe
* unit test
* fix ddpg2
* changes
* wip
* args
* faster test
* common
* fix
* add alg option
* batch mode and policy serving
* multi serving test
* todo
* wip
* serving test
* doc async env
* num envs
* comments
* thread
* remove init hook
* update
* fix ppo
* comments1
* fix
* updates
* add jenkins tests
* fix
* fix pytorch
* fix
* fixes
* fix a3c policy
* fix squeeze
* fix trunc on apex
* fix squeezing for real
* update
* remove horizon test for now
* multiagent wip
* update
* fix race condition
* fix ma
* t
* doc
* st
* wip
* example
* wip
* working
* cartpole
* wip
* batch wip
* fix bug
* make other_batches None default
* working
* debug
* nit
* warn
* comments
* fix ppo
* fix obs filter
* update
* wip
* tf
* update
* fix
* cleanup
* cleanup
* spacing
* model
* fix
* dqn
* fix ddpg
* doc
* keep names
* update
* fix
* com
* docs
* clarify model outputs
* Update torch_policy_graph.py
* fix obs filter
* pass thru worker index
* fix
* rename
* vlad torch comments
* fix log action
* debug name
* fix lstm
* remove unused ddpg net
* remove conv net
* revert lstm
* wip
* wip
* cast
* wip
* works
* fix a3c
* works
* lstm util test
* doc
* clean up
* update
* fix lstm check
* move to end
* fix sphinx
* fix cmd
* remove bad doc
* envs
* vec
* doc prep
* models
* rl
* alg
* up
* clarify
* copy
* async sa
* fix
* comments
* fix a3c conf
* tune lstm
* fix reshape
* fix
* back to 16
* tuned a3c update
* update
* tuned
* optional
* merge
* wip
* fix up
* move pg class
* rename env
* wip
* update
* tip
* alg
* readme
* fix catalog
* readme
* doc
* context
* remove prep
* comma
* add env
* link to paper
* paper
* update
* rnn
* update
* wip
* clean up ev creation
* fix
* fix
* fix
* fix lint
* up
* no comma
* ma
* Update run_multi_node_tests.sh
* fix
* sphinx is stupid
* sphinx is stupid
* clarify torch graph
* no horizon
* fix config
* sb
* Update test_optimizers.py
2018-07-01 00:05:08 -07:00
