some results

This commit is contained in:
wassname
2017-11-18 08:39:48 +08:00
parent 2700556b59
commit 9685971602
6 changed files with 723 additions and 159 deletions
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 105 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 167 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 110 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 186 KiB

+711 -153
View File
File diff suppressed because one or more lines are too long
+12 -6
View File
@@ -1,20 +1,26 @@
MNIST is the hello world of machine learning. When we learn out optimizer moves through peaks and valleys. But how bumpy are those valleys and how many are there?
This project tries to map a slice of the MNIST classification problem. We reduce mnist to a binary problem: is the number > 5?
This project tries to map a slice of the MNIST classification problem. We reduce mnist to a binary problem: is the number > 5?
- Then we pre-train a model
- freeze all but two parameters. We call those x and y.
- Height, or z, is our loss.
- freeze all but two parameters. We call those x and y.
- Height, or z, is our loss.
- Then we peform a grid search over x and y to find the loss at differen't points.
- finally we countour the landscape
- and show the path of our model as it learns
- and show the path of our model as it learns
Results:
![](docs/countours.png)
![](docs/loss.png)
![](docs/logloss.png)
![](docs/std.png)
![](docs/logstd.png)
TODO
- [ ] run a finer grid
- [x] run a finer grid
- [ ] run a wider search
- [ ] try on an untrained model
- [ ] show path of a model as it learns
- [ ] make gif