wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-07-15 11:25:40 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,715 Commits 36 Branches 42 Tags
54215ff287c034fe17c2aa1de5a8a67dc008dca1
Commit Graph
50 Commits
Author SHA1 Message Date
architkulkarniandGitHub 6ae9e76b81 [RLlib] Fix seeding issue (#10589) 2020-09-04 17:17:53 -07:00
Eric LiangandGitHub 2a204260a8 [api] Second round of 1.0 API changes: exceptions, num_return_vals (#10377) 2020-08-28 19:57:02 -07:00
Eric LiangandGitHub 519354a39a [api] Initial API deprecations for Ray 1.0 (#10325) 2020-08-28 15:03:50 -07:00
Sven MikaandGitHub e968b52cb7 [RLlib] Trajectory view API - 03 Fast LSTM + prev actions/rewards (#9950) 2020-08-21 12:35:16 +02:00
Sven MikaandGitHub aeb5be7733 [RLlib] Trajectory View API (part 2.5): Actual implementations (not used yet) of a SampleCollector. (#10112) 2020-08-15 15:09:00 +02:00
Sven MikaandGitHub 2256047876 [RLlib] Rename rllib.utils.types into typing to match built-in python module's name. (#10114) 2020-08-15 13:24:22 +02:00
yncxcwGitHubRichard Liaw
 		32cd94b750 [Core] Do not convert gpu id to int (#9744) 
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2020-08-11 12:09:46 -07:00
Michael LuoandGitHub 4d7bd8c892 [RLlib] Implementation of "Model-based Meta Policy Optimization" (MB MPO) (#9409) 2020-08-02 18:12:09 +02:00
Sven MikaandGitHub b0b0463161 [RLlib] Trajectory View API (preparatory cleanup and enhancements). (#9678) 2020-07-29 21:15:09 +02:00
Sven MikaandGitHub e4c5d3526f Issue 9631: Tf1.14 does not have tf.config.list_physical_devices. (#9681) 2020-07-24 21:48:58 +02:00
Sven MikaandGitHub 8204717eed [RLlib] Issue 9218: PyTorch Policy places Model on GPU even with num_gpus=0 (#9516) 2020-07-17 05:53:25 +02:00
Sven MikaandGitHub 03ab86567f [RLlib] Layout of Trajectory View API (new class: Trajectory; not used yet). (#9269) 2020-07-14 04:27:49 +02:00
Sven MikaandGitHub fcdf410ae1 [RLlib] Tf2.x native. (#8752) 2020-07-11 22:06:35 +02:00
Sven MikaandGitHub 43043ee4d5 [RLlib] Tf2x preparation; part 2 (upgrading try_import_tf()). (#9136) 
* WIP.

* Fixes.

* LINT.

* WIP.

* WIP.

* Fixes.

* Fixes.

* Fixes.

* Fixes.

* WIP.

* Fixes.

* Test

* Fix.

* Fixes and LINT.

* Fixes and LINT.

* LINT.
 2020-06-30 10:13:20 +02:00
Eric LiangandGitHub 1e0e1a45e6 [rllib] Add type annotations for evaluation/, env/ packages (#9003) 2020-06-19 13:09:05 -07:00
Ian RodneyandGitHub 2e972c2a77 RLLIB and pylintrc (#8995) 2020-06-17 18:14:25 +02:00
Sven MikaandGitHub 7008902cff [RLlib] Minor rllib.utils cleanup. (#8932) 2020-06-16 08:52:20 +02:00
mehrdadnandGitHub f93bb008bb Change os.uname()[1] and socket.gethostname() to the portable and faster platform.node_ip() (#8839) 
Co-authored-by: Mehrdad <noreply@github.com>
 2020-06-08 21:29:46 -07:00
Sven MikaandGitHub 368088be85 [RLlib] Sample batch docs and cleanup. (#8778) 2020-06-04 22:47:32 +02:00
Sven MikaandGitHub d8a081a185 [RLlib] Unity3D integration (n Unity3D clients vs learning server). (#8590) 2020-05-30 22:48:34 +02:00
Sven MikaandGitHub 2746fc0476 [RLlib] Auto-framework, retire use_pytorch in favor of framework=... (#8520) 2020-05-27 16:19:13 +02:00
Eric LiangandGitHub 9a83908c46 [rllib] Deprecate policy optimizers (#8345) 2020-05-21 10:16:18 -07:00
Eric LiangandGitHub 9d012626e5 [rllib] Distributed exec workflow for impala (#8321) 2020-05-11 20:24:43 -07:00
Eric LiangandGitHub f48da50e1c [rllib] observation function api for multi-agent (#8236) 2020-05-04 22:13:49 -07:00
roireshefandGitHub dbcad35022 [RLlib] Added DefaultCallbacks which replaces old callbacks dict interface (#6972) 2020-04-16 16:06:42 -07:00
Xianyang LiuGitHubEric Liang
 		e1d3f7eba6 [rllib]Add config for rllib to support set python environments (#8026) 
* support set extra python environments

* wrap value with str

* Apply suggestions from code review

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* addresses comments

* fix lint errors

* remove unrelated changes due to format.sh

* remove unrelated changes due to format.sh

Co-authored-by: Eric Liang <ekhliang@gmail.com>
 2020-04-16 01:13:45 -07:00
Sven MikaandGitHub 1b31c11806 [RLlib] DDPG re-factor to fit into RLlib's functional algorithm builder API. (#7934) 2020-04-09 14:04:21 -07:00
Sven MikaGitHubEric Liang
 		22ccc43670 [RLlib] DQN torch version. (#7597) 
* Fix.

* Rollback.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Fix.

* Fix.

* Fix.

* Fix.

* Fix.

* WIP.

* WIP.

* Fix.

* Test case fixes.

* Test case fixes and LINT.

* Test case fixes and LINT.

* Rollback.

* WIP.

* WIP.

* Test case fixes.

* Fix.

* Fix.

* Fix.

* Add regression test for DQN w/ param noise.

* Fixes and LINT.

* Fixes and LINT.

* Fixes and LINT.

* Fixes and LINT.

* Fixes and LINT.

* Comment

* Regression test case.

* WIP.

* WIP.

* LINT.

* LINT.

* WIP.

* Fix.

* Fix.

* Fix.

* LINT.

* Fix (SAC does currently not support eager).

* Fix.

* WIP.

* LINT.

* Update rllib/evaluation/sampler.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/evaluation/sampler.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/utils/exploration/exploration.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/utils/exploration/exploration.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* WIP.

* WIP.

* Fix.

* LINT.

* LINT.

* Fix and LINT.

* WIP.

* WIP.

* WIP.

* WIP.

* Fix.

* LINT.

* Fix.

* Fix and LINT.

* Update rllib/utils/exploration/exploration.py

* Update rllib/policy/dynamic_tf_policy.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/policy/dynamic_tf_policy.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Update rllib/policy/dynamic_tf_policy.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Fixes.

* WIP.

* LINT.

* Fixes and LINT.

* LINT and fixes.

* LINT.

* Move action_dist back into torch extra_action_out_fn and LINT.

* Working SimpleQ learning cartpole on both torch AND tf.

* Working Rainbow learning cartpole on tf.

* Working Rainbow learning cartpole on tf.

* WIP.

* LINT.

* LINT.

* Update docs and add torch to APEX test.

* LINT.

* Fix.

* LINT.

* Fix.

* Fix.

* Fix and docstrings.

* Fix broken RLlib tests in master.

* Split BAZEL learning tests into cartpole and pendulum (reached the 60min barrier).

* Fix error_outputs option in BAZEL for RLlib regression tests.

* Fix.

* Tune param-noise tests.

* LINT.

* Fix.

* Fix.

* test

* test

* test

* Fix.

* Fix.

* WIP.

* WIP.

* WIP.

* WIP.

* LINT.

* WIP.

Co-authored-by: Eric Liang <ekhliang@gmail.com>
 2020-04-06 11:56:16 -07:00
Sven MikaandGitHub 1138f2ebed [RLlib] Issue 7046 cannot restore keras model from h5 file. (#7482) 2020-03-23 12:19:30 -07:00
Eric LiangandGitHub 9392cdbf74 [rllib] Add high-performance external application connector (#7641) 2020-03-20 12:43:57 -07:00
Eric LiangandGitHub dd70720578 [rllib] Rename sample_batch_size => rollout_fragment_length (#7503) 
* bulk rename

* deprecation warn

* update doc

* update fig

* line length

* rename

* make pytest comptaible

* fix test

* fi sys

* rename

* wip

* fix more

* lint

* update svg

* comments

* lint

* fix use of batch steps
 2020-03-14 12:05:04 -07:00
Eric LiangandGitHub a644060daa [rllib] First pass at pipeline implementation of DQN (#7433) 
* wip iters

* add test

* speed up

* update docs

* document it

* support serial sampling

* add test

* spacing

* annotate it

* update

* rename to pipeline

* comment

* iter2 wip

* update

* update

* context test

* update

* fix

* fix

* a3c pipeline

* doc

* update

* move timer

* comment

* add piepline test

* fix

* clean up

* document

* iter s

* wip dqn

* wip

* wip

* metrics

* metrics rename

* metrics ctx

* wip

* constants

* add todo

* suppport .union

* wip

* support union

* remove prints

* add todo

* remove auto timer

* fix up

* fix pipeline test

* typing

* fix breakage

* remove bad assert

* wip

* fix multiagent example

* fixapply

* update a3c

* remove a2c pl

* 0 workers

* wip

* wip

* share metrics

* wip

* wip

* doc

* fix weight sync and global var updates

* mode

* fix

* fix

* doc

* fix
 2020-03-07 14:47:58 -08:00
Sven MikaandGitHub 357232d124 [Core/RLlib] Move log_once from rllib to ray.util. (#7273) 
* Move log_once from rllib to tune.

* Move log_once from rllib to tune.

* LINT.

* Move to ray.util.debug.
 2020-02-27 10:40:44 -08:00
Eric LiangandGitHub 46af992efd [rllib] [experimental] custom RL training pipelines (PG_pl, A2C_pl) (#7213) 2020-02-19 16:07:37 -08:00
Eric LiangandGitHub 5df801605e Add ray.util package and move libraries from experimental (#7100) 2020-02-18 13:43:19 -08:00
Eric LiangandGitHub 026f6884b5 [rllib] Add Decentralized DDPPO trainer and documentation (#7088) 2020-02-10 15:28:27 -08:00
Sven MikaandGitHub 6e1c3ea824 [RLlib] Exploration API (+EpsilonGreedy sub-class). (#6974) 2020-02-10 15:22:07 -08:00
Eric LiangandRichard Liaw 2fb53396ad [rllib] [experimental] Decentralized Distributed PPO for torch (DD-PPO) (#6918) 2020-01-25 22:36:43 -08:00
SvenandEric Liang 60d4d5e1aa Remove future imports (#6724) 
* Remove all __future__ imports from RLlib.

* Remove (object) again from tf_run_builder.py::TFRunBuilder.

* Fix 2xLINT warnings.

* Fix broken appo_policy import (must be appo_tf_policy)

* Remove future imports from all other ray files (not just RLlib).

* Remove future imports from all other ray files (not just RLlib).

* Remove future import blocks that contain `unicode_literals` as well.
Revert appo_tf_policy.py to appo_policy.py (belongs to another PR).

* Add two empty lines before Schedule class.

* Put back __future__ imports into determine_tests_to_run.py. Fails otherwise on a py2/print related error.
 2020-01-09 00:15:48 -08:00
SvenandEric Liang f1b56fa5ee PG unify/cleanup tf vs torch and PG functionality test cases (tf + torch). (#6650) 
* Unifying the code for PGTrainer/Policy wrt tf vs torch.
Adding loss function test cases for the PGAgent (confirm equivalence of tf and torch).

* Fix LINT line-len errors.

* Fix LINT errors.

* Fix `tf_pg_policy` imports (formerly: `pg_policy`).

* Rename tf_pg_... into pg_tf_... following <alg>_<framework>_... convention, where ...=policy/loss/agent/trainer.
Retire `PGAgent` class (use PGTrainer instead).

* - Move PG test into agents/pg/tests directory.
- All test cases will be located near the classes that are tested and
  then built into the Bazel/Travis test suite.

* Moved post_process_advantages into pg.py (from pg_tf_policy.py), b/c
the function is not a tf-specific one.

* Fix remaining import errors for agents/pg/...

* Fix circular dependency in pg imports.

* Add pg tests to Jenkins test suite.
 2020-01-02 16:08:03 -08:00
SvenandEric Liang 8b16847c02 Get utils ready for better Agent torch support. (#6561) 2019-12-30 12:27:32 -08:00
Eric LiangandGitHub 2530eb90dc Move tf.test.is_gpu_available() to after session init (#6515) 
* move to after session init

* script fixes
 2019-12-17 14:55:39 -08:00
Eric LiangandGitHub 4c6739476b [rllib] Raise an error if GPUs are enabled but not tf.test.is_gpu_available() (#6365) 2019-12-05 10:13:54 -08:00
Eric LiangandGitHub ddc8855f41 Fix wrap (#6293) 2019-11-26 17:47:47 -08:00
gehringandEric Liang 8903bcd0c3 [rllib] Tracing for eager tensorflow policies with tf.function (#5705) 
* Added tracing of eager policies with `tf.function`

* lint

* add config option

* add docs

* wip

* tracing now works with a3c

* typo

* none

* file doc

* returns

* syntax error

* syntax error
 2019-09-17 01:44:20 -07:00
Eric LiangandGitHub daf38c8723 [tune] Deprecate tune.function (#5601) 
* remove tune function

* remove examples

* Update tune-usage.rst
 2019-08-31 16:00:10 -07:00
Eric LiangandGitHub 03a1b75852 [rllib] Fix some eager execution regressions with 1.13 (#5537) 
* fix bugs with 1.13

* allow disable
 2019-08-26 23:23:35 -07:00
gehringandEric Liang b520f6141e [rllib] Adds eager support with a generic TFEagerPolicy class (#5436) 2019-08-23 14:21:11 +08:00
Eric LiangandRobert Nishihara e2e30ca507 Ray, Tune, and RLlib support for memory, object_store_memory options (#5226) 2019-08-21 23:01:10 -07:00
Eric LiangandGitHub 5d7afe8092 [rllib] Try moving RLlib to top level dir (#5324) 2019-08-05 23:25:49 -07:00