Run-Skeleton-Run/trying_ddpg_with_implicit_dynamics.ipynb at 0de68133cf1a23d661081d78104deb1900dd0aef

mirror of https://github.com/wassname/Run-Skeleton-Run.git synced 2026-07-27 11:18:53 +08:00

Files

T

wassname 0de68133cf ddpg with a dynamics model

I'm trying a dynamics model to provide additional supervision. I'm using
this repo because it's performance tested on a competition and is in
pytorch. I'm intially testing with pendulum. Code is messy as it's a one
time experiment.

2018-01-18 16:41:20 +08:00

7.9 KiB

Raw Blame History

In [1]:

import os
os.environ['CUDA_VISIBLE_DEVICES']=""
os.environ["PYTHONPATH"]='.'

In [2]:

%pylab --no-import-all inline
%reload_ext autoreload
%autoreload 2

Populating the interactive namespace from numpy and matplotlib

In [3]:

os.sys.argv="ddpg/train.py --logdir ./logs_ddpg --num-threads 1 --ddpg-wrapper --skip-frames 5 --fail-reward -0.2 --reward-scale 1 --flip-state-action --actor-layers 64-64 --actor-layer-norm --actor-parameters-noise --actor-lr 0.001 --actor-lr-end 0.00001 --critic-layers 64-32 --critic-layer-norm --critic-lr 0.002 --critic-lr-end 0.00001 --initial-epsilon 0.5 --final-epsilon 0.001 --tau 0.0001".split(" ")
os.sys.argv

Out [3]:

['ddpg/train.py',
 '--logdir',
 './logs_ddpg',
 '--num-threads',
 '1',
 '--ddpg-wrapper',
 '--skip-frames',
 '5',
 '--fail-reward',
 '-0.2',
 '--reward-scale',
 '1',
 '--flip-state-action',
 '--actor-layers',
 '64-64',
 '--actor-layer-norm',
 '--actor-parameters-noise',
 '--actor-lr',
 '0.001',
 '--actor-lr-end',
 '0.00001',
 '--critic-layers',
 '64-32',
 '--critic-layer-norm',
 '--critic-lr',
 '0.002',
 '--critic-lr-end',
 '0.00001',
 '--initial-epsilon',
 '0.5',
 '--final-epsilon',
 '0.001',
 '--tau',
 '0.0001']

In [ ]:

from ddpg.train import *

In [ ]:

os.environ['OMP_NUM_THREADS'] = '1'
torch.set_num_threads(1)
args = parse_args()
train(args,
      create_model,
      create_act_update_fns,
      train_multi_thread,
      train_single_thread,
      play_single_thread)

[2018-01-18 16:39:02,792] Making new env: Pendulum-v0
[2018-01-18 16:39:02,886] Making new env: Pendulum-v0
[2018-01-18 16:39:02,889] Making new env: Pendulum-v0

Base (
  (feature_net): LinearNet (
    (net): Sequential (
      (linear_0): NoisyLinear (3 -> 64)
      (layer_norm_0): LayerNorm (
      )
      (act_0): ReLU ()
      (linear_1): NoisyLinear (64 -> 64)
      (layer_norm_1): LayerNorm (
      )
      (act_1): ReLU ()
    )
  )
)
ActorHead (
  (base): Base (
    (feature_net): LinearNet (
      (net): Sequential (
        (linear_0): NoisyLinear (3 -> 64)
        (layer_norm_0): LayerNorm (
        )
        (act_0): ReLU ()
        (linear_1): NoisyLinear (64 -> 64)
        (layer_norm_1): LayerNorm (
        )
        (act_1): ReLU ()
      )
    )
  )
  (policy_net): LinearNet (
    (net): Sequential (
      (linear_0): Linear (64 -> 1)
      (act_0): Tanh ()
    )
  )
)
CriticHead (
  (base): Base (
    (feature_net): LinearNet (
      (net): Sequential (
        (linear_0): NoisyLinear (3 -> 64)
        (layer_norm_0): LayerNorm (
        )
        (act_0): ReLU ()
        (linear_1): NoisyLinear (64 -> 64)
        (layer_norm_1): LayerNorm (
        )
        (act_1): ReLU ()
      )
    )
  )
  (value_net): Linear (64 -> 1)
)
DynamicsHead (
  (base): Base (
    (feature_net): LinearNet (
      (net): Sequential (
        (linear_0): NoisyLinear (3 -> 64)
        (layer_norm_0): LayerNorm (
        )
        (act_0): ReLU ()
        (linear_1): NoisyLinear (64 -> 64)
        (layer_norm_1): LayerNorm (
        )
        (act_1): ReLU ()
      )
    )
  )
  (value_net): Linear (65 -> 3)
)

In [ ]:

7.9 KiB Raw Blame History

7.9 KiB

Raw Blame History