wassname/viz_torch_optim
1
0
Fork 0
mirror of https://github.com/wassname/viz_torch_optim.git synced 2026-06-27 18:08:02 +08:00
Code Issues Packages Projects Releases Wiki Activity

readme

Browse Source
This commit is contained in:
wassname
2017-11-17 14:01:25 +08:00
parent 4fc3acca1e
commit d494676466
2 changed files with 378 additions and 318 deletions
Download Patch File Download Diff File Expand all files Collapse all files
main.ipynb
+374 -317
View File
File diff suppressed because one or more lines are too long
readme.md
+4 -1
View File
@@ -4,11 +4,14 @@ This project generates animations of pytorch optimizers solving toy problems. Ex
# Examples
Please note that these don't have the same learning rate as simple optimizers perform better on low dimensional problems. So with a constant learning rate SGD gets to the minima too fast to see while Adam crawls along. So I used differen't learning rate per optimizer to show them in the same video.
Please note each optimizer has a differen't learning rate. This is because simpler optimizers perform better on low dimensional problems. So, with a constant learning rate, the simpler SGD optimizer races while Adam crawls along. In that case we would be able to see SGD's path or Adam's movement. So I used differen't learning rates for each optimizer in order to show them on the same video.
## With cyclic annealing:
![](docs/videos/beales_CyclicLR_20171117_04-01-14_2d.gif)
![](docs/videos/beales_CyclicLR_20171117_04-51-12_loss.gif)
![](docs/videos/beales_CyclicLR_20171117_04-51-12_3d.gif)
## Constant learning rate