This repository will implement the classic deep reinforcement learning algorithms. The aim of this repository is to provide clear code for people to learn the deep reinforcement learning algorithm.
In the future, more algorithms will be added and the existing codes will also be maintained.
Installation
- install the pytorch
plase go to official webisite to install it: https://pytorch.org/
Recommend use Anaconda Virtual Environment to manage your packages
- install openai-baselines (the openai-baselines update so quickly, please use the older version as blow, will solve in the future.)
# clone the openai baselines
git clone https://github.com/openai/baselines.git
cd baselines
git checkout 366f486
pip install -e .
DQN
包含两个实现, SARSA实现和Q-learning实现。
SARSA是on-policy实现，因为没有历史经验。
Q-learning是off-policy实现，因为使用了历史经验。
Policy Gradient
使用下面的命令可以运行已经保存好的模型
python Run_Model.py
使用下面的命令开始训练
pytorch_MountainCar-v0.py
policyNet.pkl
这个是已经保存好的模型
Actor-Critic
这是一个算法框架，Actor-Critic下面存放的是经典的REINFORCE 方法。
PPO
Proximal-Policy-Optimization
A2C
Advantage Policy Gradient，2017年有一篇文章指出，A2C和A3C性能差异并不明显。
A3C
DDPG
TRPO
Alphago zero
我将在兵棋游戏中复现Alphago zero
Timeline
- 2018/11/01 给出Resnet15Dense1 版本，该版本收敛性不好，但胜率能够达到85%.只能够找点适用
- 2018/11/12 给出Resnet12Dense3 版本，该版本第一次出现收敛、稳定趋势，胜率达到92%
- 2018/11/13 给出Resnet12Dense3-v2版本， 该版本收敛性非常稳定，胜率逐步上升。性能待观测
- 2018/11/15 经过3天的训练，给出Wargame AI —— 勾陈版本，勾陈胜率稳定在90%以上。
勾陈由 8组神经网络构成。
Papers Related to the Deep Reinforcement Learning
[1] A Brief Survey of Deep Reinforcement Learning
[2] The Beta Policy for Continuous Control Reinforcement Learning
[3] Playing Atari with Deep Reinforcement Learning
[4] Deep Reinforcement Learning with Double Q-learning
[5] Dueling Network Architectures for Deep Reinforcement Learning
[6] Continuous control with deep reinforcement learning
[7] Continuous Deep Q-Learning with Model-based Acceleration
[8] Asynchronous Methods for Deep Reinforcement Learning
[9] Trust Region Policy Optimization
[10] Proximal Policy Optimization Algorithms
[11] Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
TO DO
- DDPG
- ACER
- TRPO
- DPPO