wassname/DeepRL
1
0
Fork 0
mirror of https://github.com/wassname/DeepRL.git synced 2026-09-09 11:13:47 +08:00
Code Issues Packages Projects Releases Wiki Activity

Use logits in CategoricalActorCriticNet

Browse Source 
A really minor change. Should we input logits instead of probs into `Categorical`? It will do less converting from logits and back, leading to increased stability during training (theoretically).
This commit is contained in:
Mike Clark
2018-06-15 08:24:14 +08:00
committed by GitHub
parent a7e070077d
commit ed943a6356
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
deep_rl/network/network_heads.py
+2 -2
View File
@@ -187,9 +187,9 @@ class CategoricalActorCriticNet(nn.Module, BaseNet):
phi = self.network.phi_body(obs)
phi_a = self.network.actor_body(phi)
phi_v = self.network.critic_body(phi)
prob = F.softmax(self.network.fc_action(phi_a), dim=-1)
logits = self.network.fc_action(phi_a)
v = self.network.fc_critic(phi_v)
dist = torch.distributions.Categorical(probs=prob)
dist = torch.distributions.Categorical(logits=logits)
if action is None:
action = dist.sample()
log_prob = dist.log_prob(action).unsqueeze(-1)