Deployed db0d347 with MkDocs version: 1.0.4

LightningModule/RequiredTrainerInterface/index.html

@@ -1173,21 +1173,22 @@ This is most likely the same as your training_step. But unlike training step, th
 <pre><code class="python">def configure_optimizers(self)
 </code></pre>
 
-<p>Set up as many optimizers as you need. Normally you'd need one. But in the case of GANs or something more esoteric you might have multiple. 
-Lightning will call .backward() and .step() on each one.  If you use 16 bit precision it will also handle that.</p>
+<p>Set up as many optimizers and (optionally) learning rate schedulers as you need. Normally you'd need one. But in the case of GANs or something more esoteric you might have multiple. 
+Lightning will call .backward() and .step() on each one in every epoch.  If you use 16 bit precision it will also handle that.</p>
 <h5 id="return">Return</h5>
-<p>List - List of optimizers</p>
+<p>List or Tuple - List of optimizers with an optional second list of learning-rate schedulers</p>
 <p><strong>Example</strong></p>
 <pre><code class="python"># most cases
 def configure_optimizers(self):
-    opt = Adam(lr=0.01)
+    opt = Adam(self.parameters(), lr=0.01)
     return [opt]
 
-# gan example
+# gan example, with scheduler for discriminator
 def configure_optimizers(self):
-    generator_opt = Adam(lr=0.01)
-    disriminator_opt = Adam(lr=0.02)
-    return [generator_opt, disriminator_opt] 
+    generator_opt = Adam(self.model_gen.parameters(), lr=0.01)
+    disriminator_opt = Adam(self.model_disc.parameters(), lr=0.02)
+    discriminator_sched = CosineAnnealing(discriminator_opt, T_max=10)
+    return [generator_opt, disriminator_opt], [discriminator_sched]
 </code></pre>
 
 <hr />

Trainer/Training Loop/index.html

@@ -396,13 +396,6 @@
     Accumulated gradients
   </a>
   
-</li>
-      
-        <li class="md-nav__item">
-  <a href="#anneal-learning-rate" title="Anneal Learning rate" class="md-nav__link">
-    Anneal Learning rate
-  </a>
-  
 </li>
       
         <li class="md-nav__item">
@@ -551,13 +544,6 @@
     Accumulated gradients
   </a>
   
-</li>
-      
-        <li class="md-nav__item">
-  <a href="#anneal-learning-rate" title="Anneal Learning rate" class="md-nav__link">
-    Anneal Learning rate
-  </a>
-  
 </li>
       
         <li class="md-nav__item">
@@ -625,16 +611,6 @@
 trainer = Trainer(accumulate_grad_batches=1)
 </code></pre>
 
-<hr />
-<h4 id="anneal-learning-rate">Anneal Learning rate</h4>
-<p>Cut the learning rate by 10 at every epoch listed in this list.</p>
-<pre><code class="python"># DEFAULT (don't anneal)
-trainer = Trainer(lr_scheduler_milestones=None)
-
-# cut LR by 10 at 100, 200, and 300 epochs 
-trainer = Trainer(lr_scheduler_milestones='100, 200, 300')
-</code></pre>
-
 <hr />
 <h4 id="force-training-for-min-or-max-epochs">Force training for min or max epochs</h4>
 <p>It can be useful to force training for a minimum number of epochs or limit to a max number</p>

Trainer/index.html

@@ -544,11 +544,11 @@ trainer.fit(model)
 <p><strong>Training loop</strong>    </p>
 <ul>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#accumulated-gradients">Accumulate gradients</a></li>
-<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#anneal-learning-rate">Anneal Learning rate</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-training-for-min-or-max-epochs">Force training for min or max epochs</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-disable-early-stop">Force disable early stop</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#gradient-clipping">Gradient Clipping</a></li>
 <li><a href="hooks">Hooks</a></li>
+<li><a href="https://williamfalcon.github.io/pytorch-lightning/Pytorch-Lightning/LightningModule/#configure_optimizers">Learning rate scheduling</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Pytorch-Lightning/LightningModule/#configure_optimizers">Use multiple optimizers (like GANs)</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#set-how-much-of-the-training-set-to-check">Set how much of the training set to check (1-100%)</a></li>
 </ul>

index.html

@@ -723,11 +723,11 @@ one could be a seq-2-seq model, both (optionally) ran by the same trainer file.<
 <h6 id="training-loop">Training loop</h6>
 <ul>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#accumulated-gradients">Accumulate gradients</a></li>
-<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#anneal-learning-rate">Anneal Learning rate</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-training-for-min-or-max-epochs">Force training for min or max epochs</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-disable-early-stop">Force disable early stop</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#gradient-clipping">Gradient Clipping</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/hooks/">Hooks</a></li>
+<li><a href="https://williamfalcon.github.io/pytorch-lightning/Pytorch-Lightning/LightningModule/#configure_optimizers">Learning rate scheduling</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Pytorch-Lightning/LightningModule/#configure_optimizers">Use multiple optimizers (like GANs)</a></li>
 <li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#set-how-much-of-the-training-set-to-check">Set how much of the training set to check (1-100%)</a></li>
 </ul>