wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-06-29 18:10:13 +08:00
Code Issues Packages Projects Releases Wiki Activity

Keep parameter space noise consistent with action space noise (Fix 5173) (#5193)

Browse Source 
*  make parameter space noise consistent with action space noise

*  modified according to lint check

*  indent
This commit is contained in:
Jones Wong
2019-07-15 03:20:35 +08:00
committed by Eric Liang
parent 322b5166ad
commit 5b13a7eb90
1 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/ray/rllib/agents/ddpg/ddpg_policy.py
+4 -1
View File
@@ -58,7 +58,10 @@ class DDPGPostprocessing(object):
distance_in_action_space = np.sqrt(
np.mean(np.square(clean_actions - noisy_actions)))
self.pi_distance = distance_in_action_space
if distance_in_action_space < self.config["exploration_ou_sigma"]:
if distance_in_action_space < \
self.config["exploration_ou_sigma"] * self.cur_noise_scale:
# multiplying the sampled OU noise by noise scale is
# equivalent to multiplying the sigma of OU by noise scale
self.parameter_noise_sigma_val *= 1.01
else:
self.parameter_noise_sigma_val /= 1.01