wassname/viz_torch_optim
1
0
Fork 0
mirror of https://github.com/wassname/viz_torch_optim.git synced 2026-06-27 16:47:48 +08:00
Code Issues Packages Projects Releases Wiki Activity

add gifs with noise

Browse Source
This commit is contained in:
wassname
2017-11-24 15:45:30 +08:00
parent 483e0f57aa
commit c0d77bfa49
4 changed files with 433 additions and 381 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/videos/beales_CyclicLR_20171124_04-24-27_2d.gif
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.5 MiB

docs/videos/beales_None_20171124_04-24-27_2d.gif
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.8 MiB

main.ipynb
+423 -381
View File
File diff suppressed because one or more lines are too long
readme.md
+10
View File
@@ -6,16 +6,26 @@ This project generates animations of pytorch optimizers solving toy problems. Ex
Please note that each optimizer has a differen't learning rate, so they are not directly comparable. This is because simpler optimizers perform better on low dimensional problems. So, with a constant learning rate, the simpler SGD optimizer races to the finish while Adam crawls along. In this case we SGD is to fast to see and Adam is too bording to watch. So I used differen't learning rates for each optimizer so I could show them on the same video. Please note that each optimizer has a differen't learning rate, so they are not directly comparable. This is because simpler optimizers perform better on low dimensional problems. So, with a constant learning rate, the simpler SGD optimizer races to the finish while Adam crawls along. In this case we SGD is to fast to see and Adam is too bording to watch. So I used differen't learning rates for each optimizer so I could show them on the same video.
## With cyclic annealing: ## With cyclic annealing:
### Beales function with noise (cyclic annealing)
![](docs/videos/beales_CyclicLR_20171117_04-51-12_2d.gif) ![](docs/videos/beales_CyclicLR_20171117_04-51-12_2d.gif)
<!-- ![](docs/videos/beales_CyclicLR_20171117_04-51-12_loss.gif) --> <!-- ![](docs/videos/beales_CyclicLR_20171117_04-51-12_loss.gif) -->
### Beales function with noise (cyclic annealing)
![](docs/videos/beales_CyclicLR_20171124_04-24-27_2d.gif)
## Constant learning rate ## Constant learning rate
### Beales function ### Beales function
![](docs/videos/beales_20171117_00-02-20_2d.gif) ![](docs/videos/beales_20171117_00-02-20_2d.gif)
## Beales function with noise
![](docs/videos/beales_None_20171124_04-24-27_2d.gif)
### Six humped camel function ### Six humped camel function
![](docs/videos/six_humped_camel_back_20171115_09-38-57.gif) ![](docs/videos/six_humped_camel_back_20171115_09-38-57.gif)
![](docs/videos/six_humped_camel_back_20171115_09-38-57_3d.gif) ![](docs/videos/six_humped_camel_back_20171115_09-38-57_3d.gif)