wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-12 12:20:11 +08:00
Code Issues Packages Projects Releases Wiki Activity

[rllib] Print out intermediate data shapes on the first iteration (#4426)

Browse Source
This commit is contained in:
Eric Liang
2019-03-26 00:27:59 -07:00
committed by GitHub
parent 8ee240f40e
commit cff08e19ff
16 changed files with 236 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/rllib-training.rst
+2 -2
View File
@@ -184,7 +184,7 @@ Accessing Policy State
~~~~~~~~~~~~~~~~~~~~~~
It is common to need to access an agent's internal state, e.g., to set or get internal weights. In RLlib an agent's state is replicated across multiple *policy evaluators* (Ray actors) in the cluster. However, you can easily get and update this state between calls to ``train()`` via ``agent.optimizer.foreach_evaluator()`` or ``agent.optimizer.foreach_evaluator_with_index()``. These functions take a lambda function that is applied with the evaluator as an arg. You can also return values from these functions and those will be returned as a list.
You can also access just the "master" copy of the agent state through ``agent.get_policy()`` or ``agent.local_evaluator``, but note that updates here may not be immediately reflected in remote replicas if you have configured ``num_workers > 0``. For example, to access the weights of a local TF policy, you can run ``agent.get_policy().get_weights()``. This is also equivalent to ``agent.local_evaluator.policy_map["default"].get_weights()``:
You can also access just the "master" copy of the agent state through ``agent.get_policy()`` or ``agent.local_evaluator``, but note that updates here may not be immediately reflected in remote replicas if you have configured ``num_workers > 0``. For example, to access the weights of a local TF policy, you can run ``agent.get_policy().get_weights()``. This is also equivalent to ``agent.local_evaluator.policy_map["default_policy"].get_weights()``:
.. code-block:: python
@@ -192,7 +192,7 @@ You can also access just the "master" copy of the agent state through ``agent.ge
agent.get_policy().get_weights()
# Same as above
agent.local_evaluator.policy_map["default"].get_weights()
agent.local_evaluator.policy_map["default_policy"].get_weights()
# Get list of weights of each evaluator, including remote replicas
agent.optimizer.foreach_evaluator(lambda ev: ev.get_policy().get_weights())