wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-12 12:20:11 +08:00
Code Issues Packages Projects Releases Wiki Activity

[rllib] SAC no_done_at_end should default to False (#7594)

Browse Source 
* update

* update doc

* stochastic

* cleanu
This commit is contained in:
Eric Liang
2020-03-14 11:16:54 -07:00
committed by GitHub
parent c3a8ba399f
commit 52cf77f5a9
4 changed files with 15 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/rllib-toc.rst
+1 -1
View File
@@ -33,7 +33,7 @@ Training APIs
- `Callbacks and Custom Metrics <rllib-training.html#callbacks-and-custom-metrics>`__
- `Customized Exploration Behavior (Training and Evaluation) <rllib-training.html#customized-exploration-behavior-training-and-evaluation>`__
- `Customizing Exploration Behavior <rllib-training.html#customizing-exploration-behavior>`__
- `Customized Evaluation During Training <rllib-training.html#customized-evaluation-during-training>`__
doc/source/rllib-training.rst
+10 -8
View File
@@ -520,8 +520,8 @@ Custom metrics can be accessed and visualized like any other training result:
.. image:: custom_metric.png
Customized Exploration Behavior (Training and Evaluation)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Customizing Exploration Behavior
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RLlib offers a unified top-level API to configure and customize an agent’s
exploration behavior, including the decisions (how and whether) to sample
@@ -665,9 +665,9 @@ Customized Evaluation During Training
RLlib will report online training rewards, however in some cases you may want to compute
rewards with different settings (e.g., with exploration turned off, or on a specific set
of environment configurations). You can evaluate policies during training by setting one
or more of the ``evaluation_interval``, ``evaluation_num_episodes``, ``evaluation_config``,
``evaluation_num_workers``, and ``custom_eval_function`` configs
of environment configurations). You can evaluate policies during training by setting
the ``evaluation_interval`` config, and optionally also ``evaluation_num_episodes``,
``evaluation_config``, ``evaluation_num_workers``, and ``custom_eval_function``
(see `trainer.py <https://github.com/ray-project/ray/blob/master/rllib/agents/trainer.py>`__ for further documentation).
By default, exploration is left as-is within ``evaluation_config``.
@@ -682,9 +682,11 @@ via:
"explore": False
}
**IMPORTANT NOTE**: Policy gradient algorithms are able to find the optimal
policy, even if this is a stochastic one. Setting "explore=False" above
will result in the evaluation workers not using this optimal policy.
.. note::
Policy gradient algorithms are able to find the optimal
policy, even if this is a stochastic one. Setting "explore=False" above
will result in the evaluation workers not using this stochastic policy.
There is an end to end example of how to set up custom online evaluation in `custom_eval.py <https://github.com/ray-project/ray/blob/master/rllib/examples/custom_eval.py>`__. Note that if you only want to eval your policy at the end of training, you can set ``evaluation_interval: N``, where ``N`` is the number of training iterations before stopping.