update tutorial (#318)

This commit is contained in:
Robert Nishihara
2016-07-28 20:47:37 -07:00
committed by Philipp Moritz
parent f79494c8c6
commit 8d5e61d3c0
14 changed files with 357 additions and 339 deletions
+4 -4
View File
@@ -1,4 +1,4 @@
## Hyperparameter Optimization
# Hyperparameter Optimization
This document provides a walkthrough of the hyperparameter optimization example.
To run the application, first install this dependency.
@@ -22,7 +22,7 @@ Choosing these parameters can be challenging, and so a common practice is to
search over the space of hyperparameters. One approach that works surprisingly
well is to randomly sample different options.
### The serial version
## The serial version
Suppose that we want to train a convolutional network, but we aren't sure how to
choose the following hyperparameters:
@@ -74,7 +74,7 @@ Of course, as there are no dependencies between the different invocations of
`train_cnn_and_compute_accuracy`, this computation could easily be parallelized
over multiple cores or multiple machines. Let's do that now.
### The distributed version
## The distributed version
First, let's turn `train_cnn_and_compute_accuracy` into a remote function in Ray
by writing it as follows. In this example application, a slightly more
@@ -115,7 +115,7 @@ their values with `ray.get`.
results = [(params, ray.get(ref)) for (params, ref) in result_refs]
```
### Additional notes
## Additional notes
**Early Stopping:** Sometimes when running an optimization, it is clear early on
that the hyperparameters being used are bad (for example, the loss function may
+3 -3
View File
@@ -1,4 +1,4 @@
## Batch L-BFGS
# Batch L-BFGS
This document provides a walkthrough of the L-BFGS example. To run the
application, first install these dependencies.
@@ -21,7 +21,7 @@ one such algorithm. It is a quasi-Newton method that uses gradient information
to approximate the inverse Hessian of the loss function in a computationally
efficient manner.
### The serial version
## The serial version
First we load the data in batches. Here, each element in `batches` is a tuple
whose first component is a batch of `100` images and whose second component is a
@@ -73,7 +73,7 @@ theta_init = 1e-2 * np.random.normal(size=dim)
result = scipy.optimize.fmin_l_bfgs_b(full_loss, theta_init, fprime=full_grad)
```
### The distributed version
## The distributed version
In this example, the computation of the gradient itself can be done in parallel
on a number of workers or machines.