wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-05 13:21:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,261 Commits 36 Branches 42 Tags
7aec35750178dbec660906383203f6a1b4fbb499
Commit Graph
25 Commits
Author SHA1 Message Date
Eric LiangandGitHub 7aec357501 [rllib] Multi-GPU support for Multi-Agent PPO (#3479) 
* wip

* fix

* remove check

* fix null

* revert

* lint and kl

* also fix rollout
 2018-12-08 18:02:33 -08:00
Eric LiangandGitHub d8205976e8 [rllib] Auto clip actions to Box space range; deprecate squash_to_range (#3426) 
* fix clip

* tweak wording

* remove squash entirely

* Update rllib-models.rst

* fix argument order

* Apply suggestions from code review

Co-Authored-By: ericl <ekhliang@gmail.com>
 2018-12-03 19:55:25 -08:00
Eric LiangandRichard Liaw 18a8dbfcfb [rllib] Clip DDPG ou-noise to avoid exceeding action bounds (#3386) 
Closes #2965
 2018-11-24 00:56:50 -08:00
Eric LiangandGitHub 706dc1d473 [rllib] Add test for multi-agent support and fix IMPALA multi-agent (#3289) 
IMPALA support for multiagent was broken since IMPALA has a requirement that batch sizes be of a certain length. However multi-agent envs can create variable-length batches.

Fix this by adding zero-padding as needed (similar to the RNN case).
 2018-11-14 14:14:07 -08:00
Robert NishiharaandPhilipp Moritz 5aa29613db Fix linting errors. (#3127) 2018-10-24 16:30:00 -07:00
Eric LiangandGitHub 59901a88a0 [rllib] Native support for Dict and Tuple spaces; fix Tuple action spaces; add prev a, r to LSTM (#3051) 2018-10-20 15:21:22 -07:00
Eric LiangandGitHub a9e454f6fd [rllib] Include config dicts in the sphinx docs (#3064) 2018-10-16 15:55:11 -07:00
Eric LiangandGitHub 3c891c6ece [rllib] Parallel-data loading and multi-gpu support for IMPALA (#2766) 2018-10-15 11:02:50 -07:00
eugenevinitskyandEric Liang 1943ae44da [rllib] Use SGD optimizer for ARS (#2916) 2018-09-26 22:32:26 -07:00
Eric LiangandGitHub 995ac24a2c [rllib] clarify train batch size for PPO (#2793) 
It's possible to configure PPO in a way that ends up discarding most of the samples (they are treated as "stragglers"). Add a warning when this happens, and raise an exception if the waste is particularly egregious.
 2018-09-05 12:06:13 -07:00
Eric LiangandGitHub 01b030bd57 [rllib] throw an error for continuous action spaces in IMPALA 
We currently don't support this since the reference vtrace.py does not, though it could be an interesting extension.
 2018-09-03 11:12:55 -07:00
eugenevinitskyandEric Liang 6201a6d1c7 [rllib] add augmented random search (#2714) 
* added ars

* functioning ars with regression test

* added regression tests for ARs

* fixed default config for ARS

* ARS code runs, now time to test

* ARS working and tested, changed std deviation of meanstd filter to initialize to 1

* ARS working and tested, changed std deviation of meanstd filter to initialize to 1

* pep8 fixes

* removed unused linear model

* address comments

* more fixing comments

* post yapf

* fixed support failure

* Update LICENSE

* Update policies.py

* Update test_supported_spaces.py

* Update policies.py

* Update LICENSE

* Update test_supported_spaces.py

* Update policies.py

* Update policies.py

* Update filter.py
 2018-08-24 22:20:02 -07:00
Eric LiangandGitHub fbe6c59f72 [rllib] Misc fixes, A2C (#2679) 
A bunch of minor rllib fixes:

pull in latest baselines atari wrapper changes (and use deepmind wrapper by default)
move reward clipping to policy evaluator
add a2c variant of a3c
reduce vision network fc layer size to 256 units
switch to 84x84 images
doc tweaks
print timesteps in tune status
 2018-08-20 15:28:03 -07:00
Eric LiangandGitHub 53f9755594 [rllib] Fix support for mixed discrete and continuous action spaces, add to regression test (#2655) 
* fix

* lint

* fix
 2018-08-15 10:19:41 -07:00
Eric LiangandGitHub d01dc9e22d [rllib] format with yapf (#2427) 
* initial yapf

* manual fix yapf bugs
 2018-07-19 15:30:36 -07:00
Eric LiangandGitHub d24f19fd1e [rllib] Fix stats collection and some docs bugs since the refactoring (#2361) 
* fix

* fix pbt example

* fix

* fix

* single thread by default

* vec

* fix

* fix
 2018-07-07 13:29:20 -07:00
Eric LiangandGitHub 8aa56c12e6 [rllib] Document "v2" APIs (#2316) 
* re

* wip

* wip

* a3c working

* torch support

* pg works

* lint

* rm v2

* consumer id

* clean up pg

* clean up more

* fix python 2.7

* tf session management

* docs

* dqn wip

* fix compile

* dqn

* apex runs

* up

* impotrs

* ddpg

* quotes

* fix tests

* fix last r

* fix tests

* lint

* pass checkpoint restore

* kwar

* nits

* policy graph

* fix yapf

* com

* class

* pyt

* vectorization

* update

* test cpe

* unit test

* fix ddpg2

* changes

* wip

* args

* faster test

* common

* fix

* add alg option

* batch mode and policy serving

* multi serving test

* todo

* wip

* serving test

* doc async env

* num envs

* comments

* thread

* remove init hook

* update

* fix ppo

* comments1

* fix

* updates

* add jenkins tests

* fix

* fix pytorch

* fix

* fixes

* fix a3c policy

* fix squeeze

* fix trunc on apex

* fix squeezing for real

* update

* remove horizon test for now

* multiagent wip

* update

* fix race condition

* fix ma

* t

* doc

* st

* wip

* example

* wip

* working

* cartpole

* wip

* batch wip

* fix bug

* make other_batches None default

* working

* debug

* nit

* warn

* comments

* fix ppo

* fix obs filter

* update

* wip

* tf

* update

* fix

* cleanup

* cleanup

* spacing

* model

* fix

* dqn

* fix ddpg

* doc

* keep names

* update

* fix

* com

* docs

* clarify model outputs

* Update torch_policy_graph.py

* fix obs filter

* pass thru worker index

* fix

* rename

* vlad torch comments

* fix log action

* debug name

* fix lstm

* remove unused ddpg net

* remove conv net

* revert lstm

* wip

* wip

* cast

* wip

* works

* fix a3c

* works

* lstm util test

* doc

* clean up

* update

* fix lstm check

* move to end

* fix sphinx

* fix cmd

* remove bad doc

* envs

* vec

* doc prep

* models

* rl

* alg

* up

* clarify

* copy

* async sa

* fix

* comments

* fix a3c conf

* tune lstm

* fix reshape

* fix

* back to 16

* tuned a3c update

* update

* tuned

* optional

* merge

* wip

* fix up

* move pg class

* rename env

* wip

* update

* tip

* alg

* readme

* fix catalog

* readme

* doc

* context

* remove prep

* comma

* add env

* link to paper

* paper

* update

* rnn

* update

* wip

* clean up ev creation

* fix

* fix

* fix

* fix lint

* up

* no comma

* ma

* Update run_multi_node_tests.sh

* fix

* sphinx is stupid

* sphinx is stupid

* clarify torch graph

* no horizon

* fix config

* sb

* Update test_optimizers.py
 2018-07-01 00:05:08 -07:00
Eric LiangandGitHub 30f7c08ca7 [rllib] Remove need to pass around registry (#2250) 
* remove registry

* fix

* too many _

* fix

* cloudpickle

* Update registry.py

* yapf

* fix test

* fix kv check
 2018-06-19 22:47:00 -07:00
Eric LiangandRichard Liaw 71eb558eb0 [rllib] Refactor rllib to have a common sample collection pathway (#2149) 2018-06-09 00:21:35 -07:00
Eric LiangandGitHub 47bc4c3009 [rllib] Add DDPG documentation, rename DDPG2 <=> DDPG (#1946) 
* updates

* updates

* updates

* updates

* updates

* updates

* Update rllib.rst

* Update policy-optimizers.rst
 2018-04-30 00:18:15 -07:00
Jones WongandEric Liang c9a7744e52 [rllib] Contribute DDPG to RLlib (#1877) 
*  ongoing ddpg

*  ongoing ddpg converged

*  gpu machine changes

*  tuned

*  tuned ddpg specification

*  ddpg

*  supplement missed optimizer argument clip_rewards in default DQN configuration

*  ddpg supports vision env (atari) now

*  revised according to code review comments

*  added regression test case

*  removed irrelevant files

*  validate ddpg on mountain_car_continuous

*  restore unnecessary slight changes

*  revised according to eric's comments

*  added the requested tests

*  revised accordingly

*  revised accordingly and re-validated

*  formatted by yapf

*  fix lint errors

*  formatted by yapf

*  fix lint errors

*  formatted by yapf

*  fix lint error
 2018-04-19 22:36:29 -07:00
butchcomandRichard Liaw 936bebef99 [rllib] Upgrade to OpenAI Gym 0.10.3 (#1601) 2018-03-06 00:31:02 -08:00
Eric LiangandGitHub ecb811c26e [rllib] Ape-X implementation and DQN refactor to handle replay in policy optimizer (#1604) 
* minimal apex checkin

* cleanup dqn options

* actor utils

* Sun Feb 25 17:39:54 PST 2018

* update

* compression refactor

* fix

* add test

* fix models

* Sun Feb 25 21:46:27 PST 2018

* Wed Feb 28 10:26:34 PST 2018

* Wed Feb 28 10:28:09 PST 2018

* Wed Feb 28 10:42:59 PST 2018

* refactor

* Wed Feb 28 11:17:19 PST 2018

* Wed Feb 28 11:42:08 PST 2018

* Wed Feb 28 11:42:13 PST 2018

* Wed Feb 28 11:59:02 PST 2018

* Wed Feb 28 11:59:58 PST 2018

* Wed Feb 28 12:00:08 PST 2018

* Wed Feb 28 12:02:19 PST 2018

* Wed Feb 28 13:44:31 PST 2018

* Wed Feb 28 17:01:20 PST 2018

* Sat Mar  3 14:55:59 PST 2018

* make optimizer construction explicit

* Sat Mar  3 18:23:08 PST 2018

* Sat Mar  3 18:24:28 PST 2018

* Sat Mar  3 18:49:28 PST 2018

* Sat Mar  3 18:50:42 PST 2018

* Sat Mar  3 18:56:10 PST 2018
 2018-03-04 12:25:25 -08:00
alvkao58andRichard Liaw 81a4be8f65 [rllib] Added vanilla policy gradient (#1497) 2018-02-10 13:54:51 -08:00
Eric LiangandRichard Liaw 1d2a28ab07 [rllib] test all combinations of {obs_space} x {action_space} (#1449) 2018-01-24 11:03:43 -08:00