Eric Liang 5f430da180 [rllib] Provide internal access to episode state in compute_actions() and allow returning extra batches (#2559) ...

The goal of this PR is to allow custom policies to perform model-based rollouts. In the multi-agent setting, this requires access to not only policies of other agents, but also their current observations.
Also, you might want to return the model-based trajectories as part of the rollout for efficiency.

  compute_actions() now takes a new keyword arg episodes
  pull out internal episode class into a top-level file
  add function to return extra trajectories from an episode that will be appended to the sample batch
  documentation

2018-08-16 14:37:21 -07:00

..

_static

Move documentation to ReadTheDocs. (#326)

2017-02-27 21:14:31 -08:00

_templates

Add better analytics to docs (#1854)

2018-04-10 00:51:44 -07:00

images

[tune] Split Search from Scheduling (#2452)

2018-08-04 21:27:39 -07:00

actors.rst

Add actor reconstruction limitations to documentation (#1452)

2018-01-23 13:40:50 -08:00

apex.png

[rllib] Document "v2" APIs (#2316)

2018-07-01 00:05:08 -07:00

api.rst

ray exec and ray attach commands (#2560)

2018-08-15 14:31:50 -07:00

autoscaler-status.png

Update the pip wheel in example.yaml and add docs (#1381)

2018-01-01 13:02:05 -08:00

autoscaling.rst

ray exec and ray attach commands (#2560)

2018-08-15 14:31:50 -07:00

conf.py

Support older version TF and Support RMSProp in Impala (#2590)

2018-08-09 19:51:32 -07:00

contact.rst

Add mailing list to README and documentation. (#950)

2017-09-09 10:21:51 -07:00

development.rst

Fix yapf excludes, print diff in --all mode (#2211)

2018-06-08 02:25:55 -07:00

es.png

[rllib] Document "v2" APIs (#2316)

2018-07-01 00:05:08 -07:00

example-a3c.rst

[docs] Add backlinks from hyperopt / rl algorithm examples to the built-on Ray libraries (#1356)

2017-12-23 00:31:33 -08:00

example-cython.rst

Replace python setup.py install with pip install -e . (#1460)

2018-02-22 11:15:03 -08:00

example-evolution-strategies.rst

[tune] Support user-defined trainable functions / classes / envs with a shared object registry (#1226)

2017-11-20 17:52:43 -08:00

example-lbfgs.rst

Change Python examples in documentation to use 4 space indentation. (#736)

2017-07-16 22:19:33 -07:00

example-parameter-server.rst

Synchronous parameter server example. (#1220)

2017-11-15 17:49:31 -08:00

example-policy-gradient.rst

[docs] Add backlinks from hyperopt / rl algorithm examples to the built-on Ray libraries (#1356)

2017-12-23 00:31:33 -08:00

example-resnet.rst

Allow ResNet example to run on multiple machines. (#891)

2017-08-29 21:37:53 -07:00

example-rl-pong.rst

Change Python examples in documentation to use 4 space indentation. (#736)

2017-07-16 22:19:33 -07:00

example-streaming.rst

Add streaming MapReduce example (#1251)

2017-11-27 21:38:35 -08:00

fault-tolerance.rst

Add actor reconstruction limitations to documentation (#1452)

2018-01-23 13:40:50 -08:00

hyperband.rst

[docs] update to expose libraries + landing page (#1642)

2018-03-08 09:18:09 -08:00

impala.png

[rllib] Basic IMPALA implementation (using deepmind's reference vtrace.py) (#2504)

2018-08-01 20:53:53 -07:00

index.rst

Documentation- Basic Profiling for Ray Users (#2326)

2018-07-12 16:57:39 -07:00

install-on-docker.rst

Fix installation instructions on Ubuntu and convert md -> rst. (#389)

2017-03-24 17:33:26 -07:00

installation-troubleshooting.rst

Rebase Ray on latest arrow (remove numbuf from Ray). (#910)

2017-09-04 22:58:49 -07:00

installation.rst

Update installation instructions with psmisc to enable 'ray stop' (#2550)

2018-08-05 23:58:58 -07:00

internals-overview.rst

Change Python examples in documentation to use 4 space indentation. (#736)

2017-07-16 22:19:33 -07:00

multi-agent.svg

[rllib] Document "v2" APIs (#2316)

2018-07-01 00:05:08 -07:00

pandas_on_ray.rst

Dataframe deprecation (#2353)

2018-07-06 00:16:22 -07:00

pbt.png

[tune] clean up population based training prototype (#1478)

2018-02-02 23:03:12 -08:00

pbt.rst

Fixed attribute name in code example (#2054)

2018-05-14 01:05:06 -07:00

plasma-object-store.rst

hugepage + plasma directory support plumbing + documentation (#1030)

2017-09-30 09:56:52 -07:00

ppo.png

[rllib] Document "v2" APIs (#2316)

2018-07-01 00:05:08 -07:00

profiling.rst

Documentation- Basic Profiling for Ray Users (#2326)

2018-07-12 16:57:39 -07:00

ray-tune-parcoords.png

[tune] Fix Docs (#1469)

2018-01-25 16:39:00 -08:00

ray-tune-tensorboard.png

[tune] Documentation for Ray.tune (#1243)

2017-11-23 11:31:59 -08:00

ray-tune-viskit.png

[tune] Documentation for Ray.tune (#1243)

2017-11-23 11:31:59 -08:00

redis-memory-management.rst

Doc: redis memory management / automatic flushing. (#2344)

2018-07-05 23:44:37 -07:00

resources.rst

Update resource documentation (remove outdated limitations). (#2022)

2018-05-25 22:19:47 -07:00

rllib-algorithms.rst

[rllib] Support agent.get_action in multiagent (#2543)

2018-08-02 13:35:53 -07:00

rllib-api.svg

[rllib] Update docs with api and components overview figures (#1443)

2018-01-19 10:08:45 -08:00

rllib-components.svg

[rllib] Update docs with api and components overview figures (#1443)

2018-01-19 10:08:45 -08:00

rllib-concepts.rst

[rllib] dqn/ddpg policy customization (#2445)

2018-07-22 14:47:14 -07:00

rllib-env.rst

[rllib] Document creating an ensemble of envs; also add vector_index attribute to env config (#2513)

2018-08-01 16:29:27 -07:00

rllib-envs.svg

[rllib] Document "v2" APIs (#2316)

2018-07-01 00:05:08 -07:00

rllib-models.rst

[rllib] Provide internal access to episode state in compute_actions() and allow returning extra batches (#2559)

2018-08-16 14:37:21 -07:00

rllib-package-ref.rst

[rllib] Support the timesteps_per_batch in simple optimizer PPO mode (#2558)

2018-08-06 12:10:59 -07:00

rllib-stack.svg

[rllib] Document "v2" APIs (#2316)

2018-07-01 00:05:08 -07:00

rllib-training.rst

[rllib] Fix support for mixed discrete and continuous action spaces, add to regression test (#2655)

2018-08-15 10:19:41 -07:00

rllib.rst

[rllib] Provide internal access to episode state in compute_actions() and allow returning extra batches (#2559)

2018-08-16 14:37:21 -07:00

serialization.rst

Change Python examples in documentation to use 4 space indentation. (#736)

2017-07-16 22:19:33 -07:00

throughput.png

[rllib] Document "v2" APIs (#2316)

2018-07-01 00:05:08 -07:00

timeline.png

[minor] Use a better timeline pic in the documentation

2017-12-20 12:54:25 -08:00

troubleshooting.rst

Change Python examples in documentation to use 4 space indentation. (#736)

2017-07-16 22:19:33 -07:00

tune-config.rst

[tune] Support lambda functions in hyperparameters / tune rllib multiagent support (#2568)

2018-08-07 16:29:21 -07:00

tune.rst

[rllib, tune] TrainingResult -> Dict, Removes C408 from flake8 (#2565)

2018-08-07 12:17:44 -07:00

tutorial.rst

Change Python examples in documentation to use 4 space indentation. (#736)

2017-07-16 22:19:33 -07:00

user-profiling-timeline.gif

Documentation- Basic Profiling for Ray Users (#2326)

2018-07-12 16:57:39 -07:00

user-profiling.rst

Documentation- Basic Profiling for Ray Users (#2326)

2018-07-12 16:57:39 -07:00

using-ray-and-docker-on-a-cluster.md

Enable starting and stopping ray with "ray start" and "ray stop". (#628)

2017-06-02 20:17:48 +00:00

using-ray-on-a-cluster.rst

ray exec and ray attach commands (#2560)

2018-08-15 14:31:50 -07:00

using-ray-on-a-large-cluster.rst

ray exec and ray attach commands (#2560)

2018-08-15 14:31:50 -07:00

using-ray-with-gpus.rst

Change Python examples in documentation to use 4 space indentation. (#736)

2017-07-16 22:19:33 -07:00

using-ray-with-tensorflow.rst

[tune] Tune Documentation and expose better API (#1681)

2018-03-19 12:55:10 -07:00

webui.rst

say which port is local and which one is remote (#1591)

2018-02-25 10:19:12 -08:00